Source: https://patents.google.com/patent/US20110096942A1/en
Timestamp: 2019-12-14 05:37:03
Document Index: 583806425

Matched Legal Cases: ['Application No. 61', 'art 500', 'art 500', 'art 500', 'art 500', 'art 900', 'art 900', 'art 900', 'art 900']

US20110096942A1 - Noise suppression system and method - Google Patents
US20110096942A1
US20110096942A1 US12/897,548 US89754810A US2011096942A1 US 20110096942 A1 US20110096942 A1 US 20110096942A1 US 89754810 A US89754810 A US 89754810A US 2011096942 A1 US2011096942 A1 US 2011096942A1
US12/897,548
2009-10-23 Priority to US25447709P priority Critical
2010-10-04 Application filed by Broadcom Corp filed Critical Broadcom Corp
2010-10-04 Priority to US12/897,548 priority patent/US20110096942A1/en
2011-01-07 Assigned to BROADCOM CORPORATION reassignment BROADCOM CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: THYSSEN, JES
2011-04-28 Publication of US20110096942A1 publication Critical patent/US20110096942A1/en
230000001629 suppression Effects 0 abstract claims description title 373
230000000996 additive Effects 0 abstract claims description 106
101700084572 RSMB family Proteins 0 description 3
Systems and methods are described for applying noise suppression to one or more audio signals to generate a noise-suppressed audio signal therefrom. In a single-channel implementation, an input signal is received that comprises a desired audio signal and an additive noise signal. Noise suppression is then applied to the input signal to generate a noise-suppressed signal in a manner that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of a residual noise signal included in the noise-suppressed signal. In an alternative single-channel implementation, a plurality of sub-band signals obtained by applying a frequency conversion process to a time domain representation of an input signal is received. Noise suppression is then applied to each of the sub-band signals by passing each of the sub-band signals through a time direction filter. Multi-channel noise suppression variants are also described.
This application claims priority to U.S. Provisional Patent Application No. 61/254,477 filed Oct. 23, 2009 and entitled “Noise Suppression Framework that Considers both Speech Distortion and Unnaturalness of Residual Background Noise,” the entirety of which is incorporated by reference herein.
The invention generally relates to systems and methods that process audio signals, such as speech signals, to remove undesired noise components therefrom.
An input speech signal may be viewed as comprising both a desired speech signal (sometimes referred to as “clean speech”) and an additive background noise signal. Many conventional noise suppression techniques attempt to derive a time domain filter or a frequency domain gain function that, when applied to an appropriate representation of the input speech signal, will have the effect of attenuating or removing the additive background noise signal. However, when conventional noise suppression techniques are applied to the input speech signal, two main types of distortion will occur: (1) distortion of the desired speech signal; and (2) distortion of a residual background noise signal that remains after application of noise suppression. The distortion of the residual background noise signal mentioned here is distortion that has the effect of making the residual background noise component sound unnatural. Currently, there is no noise suppression method that takes both of these types of distortion into account explicitly when deriving the noise suppression time domain filter or frequency domain gain function. For example, the legacy Wiener filter simply attempts to minimize the error between the output of the noise suppressor and the invisible clean speech component without regard to the naturalness of the residual background noise component. What is needed, then, is an approach to noise suppression that minimizes speech distortion while also maintaining a natural residual background noise. The desired approach should be applicable to all types of audio signals.
Systems and methods are described herein for applying noise suppression to one or more input audio signals to generate a noise-suppressed audio signal therefrom. In one embodiment, an input audio signal is received that comprises a desired audio signal and an additive noise signal. Noise suppression is then applied to the input audio signal to generate a noise-suppressed signal in a manner that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of a residual noise signal included in the noise-suppressed signal.
In an alternate embodiment, a first input audio signal is received that comprises a first desired audio signal and a first additive noise signal and a second input audio signal is received that comprises a second desired audio signal and a second additive noise signal. The first input audio signal is processed to generate a first processed audio signal in a manner that is controlled by at least a parameter that specifies a degree of balance between distortion of the first desired audio signal and unnaturalness of a residual noise signal included in a noise-suppressed audio signal. The second input audio signal is processed to generate a second processed audio signal in a manner that is controlled by at least the parameter that specifies the degree of balance between distortion of the first desired audio signal and unnaturalness of the residual noise signal. The first processed audio signal and the second processed audio signal are then combined to produce the noise-suppressed audio signal.
In a further embodiment, a plurality of sub-band signals obtained by applying a frequency conversion process to a time domain representation of an input audio signal is received. Noise suppression is then applied to each of the sub-band signals by passing each of the sub-band signals through a corresponding time direction filter. In one implementation in which each sub-band signal comprises a desired audio signal and a noise signal, passing each of the sub-band signals through a corresponding time direction filter comprises passing each of the sub-band signals through a time direction filter having a response that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal included in the sub-band signal and unnaturalness of a residual noise signal included in a noise-suppressed version of the sub-band signal.
In a still further embodiment, a plurality of first sub-band signals obtained by applying a frequency conversion process to a time domain representation of a first input audio signal is received and a plurality of second sub-band signals obtained by applying a frequency conversion process to a time domain representation of a second input audio signal is received. Each of the plurality of first sub-band signals is passed through a corresponding one of a plurality of first time direction filters. Each of the plurality of second sub-band signals is passed through a corresponding one of a plurality of second time direction filters. An output from each of the plurality of first time direction filters is combined with an output from a corresponding one of the plurality of second time direction filters to generate a plurality of noise-suppressed sub-band signals.
FIG. 1 is a block diagram of a single-channel noise suppression system in accordance with an embodiment of the present invention.
FIG. 2 is a graph that illustrates shaping of a residual noise signal by a shaping filter in comparison to a flat attenuation of the residual noise signal in accordance with different embodiments of the present invention.
FIG. 3 is a block diagram of an example single-channel noise suppressor that uses a time domain filter in accordance with an embodiment of the present invention.
FIG. 4 is a block diagram of an alternate example single-channel noise suppressor that uses a time domain filter in accordance with an embodiment of the present invention.
FIG. 5 depicts a flowchart of a method for performing single-channel noise suppression in the time domain in accordance with an embodiment of the present invention.
FIG. 6 is a block diagram of a dual-channel noise suppression system in accordance with an embodiment of the present invention.
FIG. 7 is a block diagram of an example dual-channel noise suppressor that uses two time domain filters in accordance with an embodiment of the present invention.
FIG. 8 is a block diagram of an alternate example dual-channel noise suppressor that uses two time domain filters in accordance with an embodiment of the present invention.
FIG. 9 depicts a flowchart of a method for performing dual-channel noise suppression in the time domain in accordance with an embodiment of the present invention.
FIG. 10 is a block diagram of an example single-channel frequency domain noise suppressor in accordance with an embodiment of the present invention.
FIG. 11 depicts a flowchart of a method for performing single-channel noise suppression in the frequency domain in accordance with an embodiment of the present invention.
FIG. 12 is a block diagram of an example dual-channel frequency domain noise suppressor in accordance with an embodiment of the present invention.
FIG. 13 depicts a flowchart of a method for performing dual-channel noise suppression in the frequency domain in accordance with an embodiment of the present invention.
FIG. 14 is a block diagram of an example single-channel noise suppressor that utilizes a hybrid approach for performing noise suppression in accordance with an embodiment of the present invention.
FIG. 15 depicts a flowchart of an example method for performing hybrid single-channel noise suppression in accordance with an embodiment of the present invention.
FIG. 16 is a block diagram of an example dual-channel noise suppressor that utilizes a hybrid approach in accordance with an embodiment of the present invention.
FIG. 17 depicts a flowchart of an example method for performing hybrid dual-channel noise suppression in accordance with an embodiment of the present invention.
As noted in the background section above, an input speech signal may be viewed as comprising both a desired speech signal and an additive background noise signal. Many conventional noise suppression techniques attempt to derive a time domain filter or a frequency domain gain function that, when applied to an appropriate representation of the input speech signal, will have the effect of attenuating or removing the additive background noise signal. However, when conventional noise suppression techniques are applied to the input speech signal, two main types of distortion will occur: (1) distortion of the desired speech signal; and (2) distortion of a residual background noise signal that remains after application of noise suppression. The distortion of the residual background noise signal mentioned here is distortion that has the effect of making the residual background noise component sound unnatural. Currently, there is no noise suppression method that takes both of these types of distortion into account explicitly when deriving the noise suppression time domain filter or frequency domain gain function. For example, the legacy Wiener filter simply attempts to minimize the error between the output of the noise suppressor and the invisible clean speech component without regard to the naturalness of the residual background noise component.
The noise suppression systems and methods described herein have been developed to enable noise suppression to be performed in a manner that provides better control of both speech distortion and unnaturalness of residual background noise. In the following, techniques in accordance with embodiments of the present invention will be described for performing (1) single channel (i.e., single microphone) noise suppression in the time domain; (2) dual channel (i.e., dual microphone) noise suppression in the time domain; (3) single channel noise suppression in the frequency domain; (4) dual channel noise suppression in the frequency domain; (5) single channel hybrid noise suppression (i.e., noise suppression in the frequency/time domain); and (6) dual channel hybrid noise suppression. Based on the teachings provided herein, persons skilled in the relevant art(s) will be able to easily extend the dual channel implementations to M channel noise suppression.
The embodiments described herein that perform noise suppression in the time domain utilize a noise suppression filter, while the embodiments described herein that perform noise suppression in the frequency domain utilize a gain function. The embodiments described herein that perform noise suppression using a hybrid approach offer the flexibility of combining the time domain and frequency domain. This may be advantageous in practice where the noise suppression comprises part of an audio framework in which a sub-band (frequency domain) representation is available but of inadequate frequency resolution for noise suppression. As will be described herein, the hybrid solution utilizes a filter in the time direction of the sub-band signals. The sub-band signals can be the frequency points from a Fast Fourier Transform (FFT) when viewed in the time direction, or can be sub-band signals from a filter bank.
Furthermore, in accordance with certain embodiments described herein, general solutions are provided that allow for arbitrary shaping of the residual background noise as inherent part of controlling the noise suppression process. Thus, these embodiments may be thought of as providing flexibility beyond just suppressing/attenuating the background noise.
Although the foregoing described the application of noise suppression to an input speech signal comprising a desired speech component and an additive background noise component to produce a noise-suppressed speech signal that includes a residual background noise component, persons skilled in the relevant art(s) will readily appreciate that the noise suppression techniques described herein may be generally applied to any input audio signal that includes a desired audio component and an additive noise component to produce a noise-suppressed audio signal that includes a residual noise component. That is to say, embodiments of the present invention are by no means limited to the application of noise suppression to speech signals only but can instead be applied to audio signals generally.
B. Single-Channel Noise Suppression in the Time Domain in Accordance with Embodiments of the Present Invention
FIG. 1 is a high-level block diagram of a single-channel noise suppression system 100 in accordance with an embodiment of the present invention. As shown in FIG. 1, system 100 includes a noise suppressor 102 that receives a single input audio signal. The single input audio signal may be received, for example, from a single microphone or may be derived from an audio signal that is received from a single microphone. Noise suppressor 102 operates to apply noise suppression to the input audio signal to generate a noise-suppressed audio signal. The input audio signal comprises a desired audio signal and an additive noise signal. As will be discussed in more detail herein, noise suppressor 102 is configured to apply noise suppression in a manner that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and the unnaturalness of a residual noise signal included in the noise-suppressed audio signal.
Noise suppression system 100 may be implemented in any system or device that operates to process audio signals for transmission, storage and/or playback to a user. For example, noise suppression system 100 may be implemented in a telecommunications device, such as a cellular telephone or headset that processes input speech signals for subsequent transmission to a remote telecommunications device via a network, although this is merely an example. Noise suppression system 100 may be implemented in hardware using analog and/or digital circuits, in software, through the execution of instructions by one or more general purpose or special-purpose processors, or as a combination of hardware and software.
In embodiments to be described in this section, noise suppressor 102 operates to receive a time domain representation of the input audio signal and to pass the time domain representation of the input audio signal through a time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal. In the following, exemplary derivations of such a time domain filter will first be described. An exemplary implementation of noise suppressor 102 that utilizes such a time domain filter will then be described. Finally, exemplary methods for performing single-channel noise suppression in the time domain will be described.
1. Example Derivation of Time Domain Filter for Single-Channel Noise Suppression
The input audio signal received by noise suppressor 102 may be represented as
y(n)=x(n)+s(n) (1)
wherein x(n) is a desired audio signal and s(n) is an additive noise signal. In a like manner to that used to derive the well-known Wiener filter, an estimate of the desired audio signal x(n) is predicted from the input audio signal y(n) by means of a finite impulse response (FIR) filter:
x ^  ( n ) = ∑ k = 0 K  h  ( k )  y  ( n - k ) ( 2 )
wherein h(k) is the impulse response, and is the entity to be estimated.
Following the classical Wiener filter analysis, the error of the estimate of the desired audio signal x(n) is analyzed,
e  ( n ) = x  ( n ) - x ^  ( n ) = x  ( n ) - ∑ k = 0 K  h  ( k )  y  ( n - k ) = x  ( n ) - ∑ k = 0 K  h  ( k )  ( x  ( n - k ) + s  ( n - k ) ) = x  ( n ) - ∑ k = 0 K  h  ( k )  x  ( n - k ) - ∑ k = 0 K  h  ( k )  s  ( n - k ) ( 3 )
wherein the observation of breaking the error term into two components originating from the desired audio signal x(n) and the additive noise signal s(n) was first seen in J. C. Chen et al., “A Minimum Distortion Noise Reduction Algorithm with Multiple Microphones,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 16, No. 3, pp. 483-493, March 2008 (the entirety of which is incorporated by reference herein). The error originating from the desired audio signal x(n) is given by
e x  ( n ) = x  ( n ) - ∑ k = 0 K  h  ( k )  x  ( n - k ) ( 4 )
and may be denoted the distortion of the desired audio signal. The error originating from the additive noise signal s(n) is given by
e s  ( n ) = - ∑ k = 0 K  h  ( k )  s  ( n - k ) ( 5 )
and may be denoted the residual noise signal. The total error signal is given by
e(n)=e x(n)+e s(n). (6)
The classical Wiener filter analysis focuses on minimizing the energy of the error signal e(n). By assuming independence of the desired audio signal x(n) and the additive noise signal s(n), following the Wiener analysis the energy of the error of the estimate of the desired audio signal x(n) can be written as
E =  ∑ n  e 2  ( n ) =  ∑ n  ( x  ( n ) - ∑ k = 0 K  h  ( k )  y  ( n - k ) ) 2 =  ∑ n  ( y  ( n ) - s  ( n ) - ∑ k = 0 K  h  ( k )  y  ( n - k ) ) 2 =  ∑ n  y 2  ( n ) + ∑ n  s 2  ( n ) + ∑ n  ( ∑ k = 0 K  h  ( k )  y  ( n - k ) ) 2 -  2  ∑ n  ∑ k = 0 K  y  ( n )  h  ( k )  y  ( n - k ) -  2  ∑ n  ∑ k = 0 K  s  ( n )  h  ( k )  y  ( n - k ) + 2  ∑ n  y  ( n )  s  ( n ) =  ∑ n  y 2  ( n ) - ∑ n  s 2  ( n ) - 2  ∑ k = 0 K  h  ( k )  ∑ n  y  ( n )  y  ( n - k ) +  2  ∑ k = 0 K  h  ( k )  ∑ n  s  ( n )  s  ( n - k ) + ∑ n  ( ∑ k = 0 K  h  ( k )  y  ( n - k ) ) 2 ( 7 )
In vector and matrix notation, this can be written as
E=r y(0)−r s(n)−2 h T r y+2 h T r s +h T R y h (8)
R _ _ y = [ r y  ( 0 ) r y  ( 1 ) … r y  ( K ) r y  ( 1 ) r y  ( 0 ) r y  ( K - 1 ) ⋮ ⋮ ⋱ ⋮ r y  ( K ) r y  ( K - 1 ) … r y  ( 0 ) ] ( 9 ) R _ _ s = [ r s  ( 0 ) r s  ( 1 ) … r s  ( K ) r s  ( 1 ) r s  ( 0 ) r s  ( K - 1 ) ⋮ ⋮ ⋱ ⋮ r s  ( K ) r s  ( K - 1 ) … r s  ( 0 ) ] ( 10 ) r _ y = [ r y  ( 0 ) , r y  ( 1 ) , …  , r y  ( K ) ] T ( 11 ) r _ s = [ r s  ( 0 ) , r s  ( 1 ) , …  , r s  ( K ) ] T ( 12 ) r y  ( k ) = ∑ n  y  ( n )  y  ( n - k ) ( 13 ) r s  ( k ) = ∑ n  s  ( n )  s  ( n - k ) ( 14 ) h _ = [ h  ( 0 ) , h  ( 1 ) , …  , h  ( K ) ] T ( 15 )
By differentiating Equation 8 with respect to h and setting to zero the Wiener filter is derived:
∂ E ∂ h _ = - 2  r _ y + 2  r _ s + 2  R _ _ y  h _ = 0  ⇓  h _ = R _ _ y - 1  ( r _ y - r _ s ) ( 16 )
The statistics of y(n) may be estimated directly, as that is the input audio signal. In an embodiment in which the input audio signal is a speech signal, the statistics of s(n) may be estimated during non-speech segments and then be assumed to be sufficiently stationary to be valid during speech segments. This seems reasonable since many kinds of background noise are stationary. However, it may pose a limitation in performance for more non-stationary kinds of background noise.
The method proposed in the aforementioned article by J. C. Chen et al. uses the technique of Lagrange multipliers to perform a constrained optimization, wherein a constraint of zero distortion of the desired audio signal is enforced upon a minimization of the residual noise signal. For single channel noise suppression, this solution degenerates to the trivial unity filter (i.e., the output of the filter equals the input) and hence no noise suppression is achieved. That finding demonstrates nicely that for single channel noise suppression, it is only possible to achieve noise suppression at the expense of distortion of the desired audio signal.
Embodiments of the present invention described herein adopt an entirely different approach that provides a meaningful solution even for single channel noise suppression. The concept is to minimize the distortion of the desired audio signal while also maintaining a natural-sounding residual noise signal. A key factor in implementing this solution is to determine how to measure unnaturalness of the residual noise signal. However, by posing a question from a different angle, a viable solution can be formed: is it possible to formulate a cost function for minimization of the distortion of the desired audio signal that encourages a natural-sounding residual noise signal?
A multitude of cost functions can be constructed. A good cost function for minimizing the unnaturalness of the residual noise signal may be the squared sum of the difference between the residual noise signal and a scaled version of the original additive noise signal. The scaling would then correspond to specifying a desired noise attenuation factor in the noise suppression algorithm. Note that a scaled-down version of the original additive noise signal will sound perfectly natural. Accordingly, a cost function for minimizing the distortion of the desired audio signal may be
E x = ∑ n  e x 2  ( n ) ( 17 )
and a cost function for minimizing the unnaturalness of the residual noise signal may be
E s = ∑ n  ( η   s  ( n ) - e s  ( n ) ) 2 ( 18 )
wherein η is the desired noise attenuation factor. For a desired noise attenuation of 15 decibels (dB), η=10(−15/20)=0.1778.
To enable a trade-off between distortion of the desired audio signal and a specified noise attenuation factor, a weighted sum of the distortion of the desired audio signal and the measure of unnaturalness of the residual noise signal is minimized:
E=αE x+(1−α)E s (19)
wherein α may be thought of as a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of the residual noise signal. This composite cost function is minimized with respect to the noise suppression filter h(k) in a like manner to the derivation of the Wiener filter:
E =  α  ( r x  ( 0 ) + h _ T  R _ _ y  h _ - h _ T  R _ _ s  h _ - 2  h _ T  r _ y + 2  h _ T  r _ s ) +  ( 1 - α )  ( η 2  r s  ( 0 ) + 2  η  h _ T  r _ s + h _ T  R _ _ s  h _ ) =  α   r y  ( 0 ) - α   r s  ( 0 ) + η  ( 1 - α )  r s  ( 0 ) + α  h _ T  R _ _ y  h _ +  ( 1 - 2  α )  h _ T  R _ _ s  h _ - 2  α  h _ T  r _ y + 2  h _ T  r _ s  ( α + η  ( 1 - α ) ) ( 20 )
Differentiating the composite cost function with respect to h and setting it to zero yields
∂ E ∂ h _ = 2  α  R _ _ y  h _ + 2  ( 1 - 2  α )  R _ _ s  h _ - 2  α  r _ y + 2  ( α + η  ( 1 - α ) )  r _ s = 0 _  ⇓  h _ = ( α  R _ _ y + ( 1 - 2  α )  R _ _ s ) - 1  ( α  r _ y - ( η  ( 1 - α ) + α )  r _ s ) ( 21 )
Thus, h provides one example implementation of a time domain filter that can be used to perform noise suppression in accordance with an embodiment of the present invention.
It is interesting to note that by specifying infinite noise attenuation, η=0, and setting the trade-off to α=½, the solution reduces to the legacy Wiener filter. Hence, the Wiener filter may be thought of as a special case of this new approach, or conversely, this new approach may be thought of as a novel generalized form of the Wiener filter that allows for specification of a desired noise attenuation factor as well as specification of a degree of balance between distortion of the desired audio signal and unnaturalness of the residual noise signal.
As an alternative to minimizing a weighted sum of the distortion of the desired audio signal and unnaturalness of the residual noise signal, one can also perform constrained optimization. For example, one can minimize the distortion of the desired audio signal with a constraint on the unnaturalness of the residual noise signal:
h _ = arg   min h _ ′  { E x  ( h _ ′ ) }   subject   to   E s  ( h _ ) = 0 ( 22 )
by using the technique of the Lagrange multiplier, i.e., by constructing the following cost function
L 1( h,λ)=E x( h )+λE s( h ), (23)
minimizing L1(h,λ) with respect to h and λ and solving for h. Conversely, one can also minimize the unnaturalness of the residual noise signal with a constraint on the distortion of the desired audio signal:
h _ = arg   min h _ ′  { E s  ( h _ ′ ) }   subject   to   E x  ( h _ ) = 0 ( 24 )
L 2( h,λ)=E s( h )+λE x( h ) (25)
with respect to h and λ and solving for h. However, unless the constraint is linear in h, regular linear algebra techniques will not suffice to solve the system of equations. In the two Lagrange cases above it can be seen that
∂ L 1  ( h _ , λ ) ∂ λ = E s  ( h _ ) = 0   ( by   design   to   enforce   the   constraint )  ⇓  r y  ( 0 ) - r s  ( 0 ) + h _ T  R _ _ y  h _ - h _ T  R _ _ s  h _ - 2  h _ T  r _ y + 2  h _ T  r _ s = 0 ( 26 ) and ∂ L 2  ( h _ , λ ) ∂ λ = E x  ( h _ ) = 0  ⇓  η 2  r s  ( 0 ) + 2  η  h _ T  r _ s + h _ T  R _ _ s  h _ = 0 ( 27 )
respectively, are both non-linear in h, and hence more complicated to solve. Hence, it may be more practical to implement the approach of minimizing a weighted sum as proposed in Equation 19 through Equation 21. For completeness, the solutions using the Lagrange multiplier in the two constrained optimization cases above would be found by solving
∂ L 1  ( h _ , λ ) ∂ h _ = ∂ E x  ( h _ ) ∂ h _ + λ  ∂ E s  ( h _ ) ∂ h _ = 0   ∂ L 1  ( h _ , λ ) ∂ λ = E s  ( h _ ) = 0 ( 28 ) and ∂ L 2  ( h _ , λ ) ∂ h _ = ∂ E s  ( h _ ) ∂ h _ + λ  ∂ E x  ( h _ ) ∂ h _ = 0   ∂ L 2  ( h _ , λ ) ∂ λ = E x  ( h _ ) = 0 ( 29 )
respectively, with respect to h. The optimal approach to obtaining a mathematically tractable solution with the technique of the Lagrange multiplier for a constrained optimization would be to construct a constraint that is linear in h, yet perceptually meaningful in minimizing the unnaturalness of the residual noise signal, for L1(h,λ), or minimizing the distortion of the desired audio signal, for L2(h,λ).
All of the above solutions, both for the cost function as a weighted sum as well as the Lagrange cost functions, were premised on a constructed cost function that reflects unnaturalness of the residual noise signal. A practical cost function for minimizing the unnaturalness of the residual noise signal was proposed in Equation 18. For the approach that minimizes a weighted sum of the distortion of the desired audio signal and the unnaturalness of the residual noise signal to be tractable, the first order derivative of the cost function must be linear in h. For the constrained optimization approach with a constraint on the unnaturalness of the residual noise signal, the cost function must be linear in h. However, for the constrained optimization approach with a constraint on the distortion of the desired audio signal, a sufficient requirement is that the first order derivative of the cost function is linear in h, but then the constraint on the distortion of the desired audio signal must be linear in h. For the approach that minimizes the weighted sum, a generalization of the cost function allows spectral shaping of the residual noise signal. FIG. 2 depicts a graph 200 that shows an example of a shaping of the residual noise signal by
H s(z)=0.1778(1−0.8·z −1), (30)
which is represented by the line labeled 202, in comparison to a flat attenuation of η=0.1778 (15 dB), which is represented by the line labeled 204.
Allowing spectral shaping of the residual noise signal generalizes the cost function of Equation 18 to
E s = ∑ n  ( ( ∑ k s = 0 K s  h s  ( k s )  s  ( n - k s ) ) - e s  ( n ) ) 2 ( 31 )
wherein Ks is the order of the shaping filter and hs (k) are the shaping filter coefficients. The weighted sum cost function of Equation 20 generalizes to
E =  α   E x + ( 1 - α )  E s =  α  ( r y  ( 0 ) - r s  ( 0 ) + h _ T  R _ _ y  h _ - h _ T  R _ _ s  h _ - 2  h _ T  r _ y + 2  h _ T  r _ s ) +  ( 1 - α )  ( h _ s T  R _ _ s ′  h _ s + 2  h _ s T  R _ _ s ″  h _ + h _ T  R _ _ s  h _ ) ( 32 )
where h s=[hs(0),hs(1), . . . , hs(Ks)]T contains the impulse response of the shaping filter and R′s and R″s are size-adjusted versions of R s that are introduced to account for any difference between Ks and K, i.e., the difference in order between the shaping filter and the noise suppression filter. Accordingly, R s is a (K+1)×(K+1) matrix, R′s is a (Ks+1)×(Ks+1) matrix, and R″s is a (Ks+1)×(K+1) matrix, but common cells of the three matrices have identical elements. The derivative of E with respect to h is given below along with the solution for h:
∂ E ∂ h _ = α  ( 2  R _ _ y  h _ - 2  R _ _ s  h _ - 2  r _ y + 2  r _ s ) + ( 1 - α )  ( 2  R _ _ s  h _ + 2  R _ _ s ″   T  h _ s ) = 0   ⇓   h _ = ( α  R _ _ y + ( 1 - 2  α )  R _ _ s ) - 1  ( α  ( r _ y - r _ s ) - ( 1 - α )  R _ _ s ″   T  h _ s ) ( 33 )
One practical implementation uses α=0.125 for Equation 21 and Equation 33, η=0.1778 for Equation 21, and the shaping filter given by Equation 30 for Equation 33.
An alternative formulation for deriving a time domain filter for single-channel noise suppression will now be described. Having inherently defined the optimal output as the sum of the desired audio signal and a scaled or filtered version of the original additive noise signal, it seems appropriate to go back and revisit the key equation for the overall error of the noise suppression process, i.e., Equation 3. The error can be expressed as
e  ( n ) =  ( x  ( n ) + ∑ k s = 0 K s  h s  ( k s )  s  ( n - k s ) ) - x ^  ( n ) =  x  ( n ) + ∑ k s = 0 K s  h s  ( k s )  s  ( n - k s ) - ∑ k = 0 K  h  ( k )  y  ( n - k ) =  x  ( n ) + ∑ k s = 0 K s  h s  ( k s )  s  ( n - k s ) - ∑ k = 0 K  h  ( k )  ( x  ( n - k ) + s  ( n - k ) ) =  x  ( n ) - ∑ k = 0 K  h  ( k )  x  ( n - k ) + ∑ k s = 0 K s  h s  ( k s )  s  ( n - k s ) -  ∑ k = 0 K  h  ( k )  s  ( n - k ) ( 34 )
wherein {circumflex over (x)}(n) is the output of the noise suppressor, x(n) is the target for the desired audio signal, and
∑ k s = 0 K s  h s  ( k s )  s  ( n - k s )
is the target for the residual noise signal. As noted previously, the target for the residual noise signal could be a spectrally flat attenuation, i.e., hs(0)=η and hs(k)=0 for k≠0. As can be seen, the formulation of Equation 34 directly includes the cost function signals. In accordance with this formulation, the distortion of the desired audio signal is defined as
e x  ( n ) = x  ( n ) - ∑ k = 0 K  h  ( k )  x  ( n - k ) ( 35 )
(which is identical to Equation 4) and the unnaturalness of the residual noise signal is now defined as
e s  ( n ) = ∑ k s = 0 K s  h s  ( k s )  s  ( n - k s ) - ∑ k = 0 K  h  ( k )  s  ( n - k ) . ( 36 )
The effective difference is a change of sign, as can be seen by comparing Equation 36 to Equation 31 with the insertion of Equation 5.
Equivalent to Equation 19, the following error term is minimized:
E = α   E x + ( 1 - α )  E s = α  ∑ n  e x 2  ( n ) + ( 1 - α )  ∑ n  e s 2  ( n ) ( 37 )
which, with previously-introduced vector and matrix notation, may be written as
E = α  ( r y  ( 0 ) - r s  ( 0 ) + h _ T  R _ _ y  h _ - h _ T  R _ _ s  h _ - 2  h _ T  r _ y + 2  h _ T  r _ s ) = + ( 1 - α )  ( h _ s T  R _ _ s ′  h _ s - 2  h _ s T  R _ _ s ″  h _ + h _ T  R _ _ s  h _ ) ( 38 )
The similarity with Equation 32 is apparent and the derivative with respect to h is calculated and set to zero in order to solve for the optimal h:
∂ E ∂ h _ = α ( 2  R _ _ y  h _ - 2  R _ _ s  h _ - 2  r _ y + 2  r _ s ) + ( 1 - α )  ( 2  R _ _ s  h _ - 2  R _ _ s ″   T  h _ s ) = 0   ⇓  h _  = ( α   R _ _ y + ( 1 - 2  α )  R _ _ s ) - 1  ( α  ( r _ y - r _ s ) + ( 1 - α )  R _ _ s ″   T  h _ s ) ( 39 )
Similar to the previously-derived time domain filter, the Wiener solution is a special case, obtained with a parameter setting of α=0.5 and h s=0. This corresponds to infinite noise attenuation and weighing distortion of the desired audio signal and unnaturalness of the residual noise signal equally.
2. Example Single-Channel Noise Suppressor that Uses a Time Domain Filter
FIG. 3 is a block diagram of an example single-channel noise suppressor 300 that uses a time domain filter in accordance with an embodiment of the present invention. Noise suppressor 300 may comprise, for example, a particular implementation of noise suppressor 102 of system 100 as described above in reference to FIG. 1. Generally speaking, noise suppressor 300 operates to receive a time domain representation of an input audio signal that comprises a desired audio signal and an additive noise signal, to pass the time domain representation of the input audio signal through a time domain filter to generate a noise-suppressed audio signal, the time domain filter having an impulse response that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of a residual noise signal in the noise-suppressed audio signal, and to output the noise-suppressed audio signal. As shown in FIG. 3, noise suppressor 300 comprises a number of interconnected components including a statistics estimation module 302, a first parameter provider module 304, a second parameter provider module 306, a time domain filter configuration module 308, and a time domain filter 310.
Statistics estimation module 302 is configured to calculate estimates of statistics associated with the input audio signal and the additive noise signal for use by time domain filter configuration module 308 in configuring time domain filter 310. The calculation of estimates may occur on some periodic or non-periodic basis depending upon a control scheme. In an embodiment, statistics estimation module 302 estimates statistics through correlation of the time domain representation of the input audio signal and correlation of a time domain representation of the additive noise signal. For example, statistics estimation module 302 may estimate ry(k) through correlation of input audio signal y(n) as illustrated in Equation 13 and estimate rs (k) through correlation of additive noise signal s(n) as illustrated in Equation 14. These values can then be used to construct matrices R y and R s (see Equations 9 and 10) and vectors r y and r s (see Equations 11 and 12), which can then be used by time domain filter configuration module 308 to configure a time domain filter such as that represented by Equation 21.
Statistics estimation module 302 may estimate the statistics of the input audio signal and the additive noise signal across a number of segments of the input audio signal. A sliding window approach may be used to select the segments. Statistics estimation module 302 may update the estimated statistics each time a new segment (e.g., each time a new frame) of the input audio signal is received. However, this example is not intended to be limiting, and the frequency with which the statistics are updated may vary depending upon the implementation.
Statistics estimation module 302 can estimate the statistics of the received input audio signal directly. In an embodiment in which the input audio signal is a speech signal, statistics estimation module 302 may estimate the statistics of the additive noise signal during non-speech segments, premised on the assumption that the additive noise signal will be sufficiently stationary during valid speech segments. In accordance with such an embodiment, statistics estimation module 302 may include functionality that is capable of classifying segments of the input audio signal as speech or non-speech segments. Alternatively, statistics estimation module 302 may be connected to another entity that is capable of performing such a function. Of course, numerous other methods may be used to estimate the statistics of the additive noise signal.
First parameter provider module 304 is configured to obtain a value of a parameter α that specifies a degree of balance between distortion of the desired audio signal included in the input audio signal and unnaturalness of a residual noise signal included in the output noise-suppressed audio signal and to provide the value of the parameter α to time domain filter configuration module 308. By way of example only, the parameter α may be that discussed above and utilized in the time domain filter representation of Equation 21.
In one embodiment, the value of the parameter α comprises a fixed aspect of noise suppressor 300 that is determined during a design or tuning phase associated with that component. Alternatively, the value of the parameter α may be determined in response to some form of user input (e.g., responsive to user control of settings of a device that includes noise suppressor 300). In a still further embodiment, first parameter provider module 304 adaptively determines the value of the parameter α based at least in part on characteristics of the input audio signal. For example, in an embodiment in which the input audio signal comprises a speech signal, first parameter provider module 304 may vary the value of the parameter α such that an increased emphasis is placed on minimizing the distortion of the desired speech signal during speech segments and such that an increased emphasis is placed on minimizing the unnaturalness of the residual noise signal during non-speech segments. Still other adaptive schemes for setting the value of parameter α may be used.
Second parameter provider module 306 is configured to obtain a value of a parameter η that specifies an amount of attenuation to be applied to the additive noise signal included in the input audio signal and to provide the value of the parameter η to time domain filter configuration module 308. By way of example only, the parameter η may be that discussed above and utilized in the time domain filter representation of Equation 21.
In one embodiment, the value of the parameter η comprises a fixed aspect of noise suppressor 300 that is determined during a design or tuning phase associated with that component. Alternatively, the value of the parameter η may be determined in response to some form of user input (e.g., responsive to user control of settings of a device that includes noise suppressor 300). In a still further embodiment, second parameter provider module 306 adaptively determines the value of the parameter η based at least in part on characteristics of the input audio signal.
In certain embodiments, first parameter provider module 304 determines a value of the parameter α based on a current value of the parameter η. Such an embodiment takes into account that certain values of α may provide a better trade-off between distortion of the desired audio signal and unnaturalness of the residual noise signal at different levels of noise attenuation. For example, as the value of η increases (i.e., as the amount of noise attenuation is increased), it may be deemed desirable to reduce the value of the γ parameter (i.e., to place more of an emphasis on reducing the unnaturalness of the residual noise signal). This is only one example, however. A scheme that derives the value of the parameter α based on the value of the parameter η may also be useful for facilitating user control of noise suppression since controlling the amount of noise attenuation may be a more intuitive and understandable operation to a user than controlling the trade-off between distortion of the desired audio signal and unnaturalness of the residual noise signal.
Time domain filter configuration module 308 is configured to obtain estimates of statistics associated with the input audio signal and the additive noise signal from statistics estimation module 302, the value of the parameter α that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal provided by first parameter provider module 304, and the value of the parameter η that specifies the amount of attenuation to be applied to the additive noise signal provided by second parameter provider module 306 and to use those values to configure time domain filter 310. For example, time domain filter configuration module 308 may use these values to configure time domain filter 310 in accordance with Equation 21, although this is only one example. Time domain filter configuration module 308 may re-configure time domain filter 310 each time a new segment of the input audio signal is received or in accordance with some other periodic or non-periodic control scheme.
Time domain filter 310 is configured to filter the input audio signal to generate and output a noise-suppressed audio signal. As discussed above, the filtering process performed by time domain filter 310 may be controlled by the estimates of statistics associated with the input audio signal and the additive noise signal from statistics estimation module 302, the value of the parameter α that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal provided by first parameter provider module 304, and the value of the parameter η that specifies the amount of attenuation to be applied to the additive noise signal provided by second parameter provider module 306.
FIG. 4 is a block diagram of an alternate example single-channel noise suppressor 400 that uses a time domain filter in accordance with an embodiment of the present invention. Noise suppressor 400 may also comprise, for example, a particular implementation of noise suppressor 102 of system 100 as described above in reference to FIG. 1. Like noise suppressor 300, noise suppressor 400 operates to receive a time domain representation of an input audio signal that comprises a desired audio signal and an additive noise signal, to pass the time domain representation of the input audio signal through a time domain filter to generate a noise-suppressed audio signal, the time domain filter having an impulse response that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of a residual noise signal in the noise-suppressed signal, and to output the noise-suppressed audio signal.
As shown in FIG. 4, noise suppressor 400 comprises a number of interconnected components including a statistics estimation module 402, a first parameter provider module 404, a noise shaping filter provider module 406, a time domain filter configuration module 408, and a time domain filter 410. Statistics estimation module 402, first parameter provider module 404, time domain filter configuration module 408 and time domain filter 410 respectively operate in essentially the same fashion as statistics estimation module 302, first parameter provider module 304, time domain filter configuration module 308 and time domain filter 310 as described above in reference to noise suppressor 300 of FIG. 3, with exceptions to be described below.
In noise suppressor 400, noise shaping filter provider module 406 is configured to provide parameters associated with a noise shaping filter h s to time domain filter configuration module 408 for use in configuring time domain filter 410. For example, time domain filter configuration module 408 may utilize the parameters of the noise shaping filter noise shaping filter h s to configure time domain filter 410 in accordance with Equation 33 as previously described. In contrast to noise suppressor 300 which uses a noise attenuation factor η, noise suppressor 400 allows for arbitrary shaping of the residual noise signal through provision of the noise shaping filter h s. Depending upon the implementation, the noise shaping filter h s may be specified during design or tuning of a device that includes noise suppressor 400, determined based on some form of user input, or adaptively determined based on at least characteristics associated with the input audio signal.
3. Example Methods for Performing Single-Channel Noise Suppression in the Time Domain
FIG. 5 depicts a flowchart 500 of a method for performing single-channel noise suppression in the time domain in accordance with an embodiment of the present invention. The method of flowchart 500 may be performed, for example and without limitation, by noise suppressor 300 as described above in reference to FIG. 3 or noise suppressor 400 as described above in reference to FIG. 4. However, the method is not limited to those implementations.
As shown in FIG. 5, the method of flowchart 500 begins at step 502 in which a time domain representation of an input audio signal is received, wherein the input audio signal comprises a desired audio signal and an additive noise signal.
At step 504, the time domain representation of the input audio signal is passed through a time domain filter to generate a noise-suppressed audio signal, wherein the time domain filter has an impulse response that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of a residual noise signal included in the noise-suppressed audio signal. For example, the time domain filter may be either of the time domain filters represented by Equation 21 or 33 and the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal may comprise the parameter α included in those equations. However, these are examples only and other time domain filters may be used.
Depending upon the implementation, the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal may be determined in a variety of ways. For example, the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal may be determined based at least in part on characteristics of the input audio signal.
In certain embodiments, step 504 involves passing the time domain representation of the input audio signal through a time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal and a noise attenuation factor. For example, the time domain filter may be the time domain filter represented by Equation 21 and the noise attenuation factor may comprise the parameter η included in that equation. However, this is one example only and other time domain filters that include a noise attenuation factor may be used. In certain embodiments, the value of the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal is determined based on the value of the noise attenuation factor.
In other embodiments, step 504 involves passing the time domain representation of the input audio signal through a time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal and a noise shaping filter. For example, the time domain filter may be the time domain filter represented by Equation 33 and the noise shaping filter may comprise the filter h s included in that equation. However, this is one example only and other time domain filters that include a noise shaping filter may be used.
In certain implementations, the method of flowchart 500 further includes estimating statistics comprising correlation of the time domain representation of the input audio signal and correlation of a time domain representation of the additive noise signal. For example and without limitation, this estimation of statistics may comprise estimating ry (k) through correlation of input audio signal y(n) as illustrated in Equation 13 and estimating rs(k) through correlation of additive noise signal s(n) as illustrated in Equation 14. These values can then be used to construct matrices R y and R s (see Equations 9 and 10) and vectors r y and r s (see Equations 11 and 12), which can then be used to implement a time domain filter such as that represented by Equation 21 or Equation 33.
In accordance with such an implementation, step 504 may involve passing the time domain representation of the input audio signal through a time domain filter having an impulse response that is a function of at least the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal and at least some of the estimated statistics.
At step 506, the noise-suppressed audio signal generated during step 504 is output. Depending upon the implementation, the noise-suppressed audio signal may then be further processed, stored, transmitted to a remote entity, or played back to a user.
C. Dual-Channel Noise Suppression in the Time Domain in Accordance with Embodiments of the Present Invention
FIG. 6 is a high-level block diagram of a dual-channel noise suppression system 600 in accordance with an embodiment of the present invention. As shown in FIG. 6, system 600 includes a noise suppressor 602 that receives a first input audio signal and a second input audio signal. The first input audio signal comprises a first desired audio signal and a first additive noise signal while the second input audio signal comprises a second desired audio signal and a second additive noise signal. The first input audio signal may be received, for example, from a first microphone or may be derived from an audio signal that is received from a first microphone and the second input audio signal may be received, for example, from a second microphone or may be derived from an audio signal that is received from a second microphone.
As will be discussed in more detail herein, noise suppressor 602 processes the first input audio signal to generate a first processed audio signal in a manner that is controlled by at least a parameter that specifies a degree of balance between distortion of the first desired audio signal and unnaturalness of a residual noise signal included in a noise-suppressed audio signal. Noise suppressor 602 also processes the second input audio signal to generate a second processed audio signal in a manner that is controlled by at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal. Noise suppressor 602 then combines the first processed audio signal and the second processed audio signal to produce the noise-suppressed signal for output.
Noise suppression system 600 may be implemented in any system or device that operates to process audio signals for transmission, storage and/or playback to a user. For example and without limitation, noise suppression system 600 may be implemented in a telecommunications device, such as a cellular telephone or headset that processes input speech signals for subsequent transmission to a remote telecommunications device via a network, although this is merely an example. Noise suppression system 600 may be implemented in hardware using analog and/or digital circuits, in software, through the execution of instructions by one or more general purpose or special-purpose processors, or as a combination of hardware and software.
In embodiments to be described in this section, noise suppressor 602 operates to pass a time domain representation of the first input audio signal through a first time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal and to pass a time domain representation of the second input audio signal through a second time domain filter having an impulse response that is also controlled by at least the parameter that specifies the degree of balance between the distortion of the desired audio signal and the unnaturalness of the residual noise signal. In the following, exemplary derivations of the two time domain filters will first be described. An exemplary implementation of noise suppressor 602 that utilizes such time domain filters will then be described. Finally, exemplary methods for performing dual-channel noise suppression in the time domain will be described.
1. Example Derivation of Time Domain Filters for Dual-Channel Noise Suppression
With two physically disjoint observations, additional information is inherently available. Consider two microphones with outputs y1(n) and y2(n), respectively. The noise, s1(n) and s2(n), and desired audio components, x1(n) and x2(n), at the microphones are additive. Furthermore, the two desired audio signals, x1(n) and x2(n), originate from a single desired source, x(n), but due to the physical dislocation of the two microphones, the acoustic coupling between the source and the two microphones is different. The acoustic coupling is modeled by an impulse response, g1(n) and g2(n), respectively. Hence, the two observations are given by
y 1(n)=x 1(n)+s 1(n)=g 1(k)*x(n)+s 1(n)
y 2(n)=x 2(n)+s 2(n)=g 2(k)*x(n)+s 2(n) (40)
By attempting to estimate x(n), the acoustic coupling between the source and the microphones would be considered and de-reverberation would be performed. This may be advantageous since reverberation in some cases can be objectionable and decrease intelligibility and/or increase listener fatigue. It is, however, a difficult task that further complicates the problem. Furthermore, referring to traditional single channel noise suppression, the goal is commonly to estimate the desired source at the microphone (and not at the location of the source, although the two may be approximately co-located in traditional handheld telephony). To provide direct comparison to the previously-described derivation of a time domain filter for a single channel, the present treatment will aim at estimating the desired source at a microphone, and hence, the developed method will not be capable of performing any de-reverberation. Note that the idea of estimating the desired source at a microphone for multi-microphone noise suppression was previously described in J. C. Chen et al., “A Minimum Distortion Noise Reduction Algorithm with Multiple Microphones,” IEEE Transactions on Audio, Speech and Language Processing, Vol. 16, No. 3, pp. 483-493, March 2008. However, that approach has often been the common approach for single-microphone noise suppression.
Without loss of generality, the following will aim at estimating the desired source at the first microphone, i.e., at estimating x1(n). Similar to single-channel noise suppression in the time domain, this is achieved with FIR filtering, except that now two filters, h1(k1) and h2(k2), are used:
x ^ 1  ( n ) = ∑ k 1 = 0 K 1  h 1  ( k 1 )  y 1  ( n - k 1 ) + ∑ k 2 = 0 K 21  h 2  ( k 2 )  y 2  ( n - k 2 ) , ( 41 )
exploiting the signals from both microphones. The objective is to estimate
h 1 =[h 1(0),h 1(1), . . . ,h 1(K 1)]T, and (42)
h 2 =[h 2(0),h 2(1), . . . ,h 2(K 2)]T (43)
according to a suitable cost function, so that satisfactory noise suppression is achieved.
In a like manner to that shown in Equation 3, the error signal is broken into two components, distortion of the desired audio signal and residual noise, in accordance with
e  ( n ) =  x 1  ( n ) - x ^ 1  ( n ) =  x 1  ( n ) - ∑ k 1 = 0 K 1  h 1  ( k 1 )  y 1  ( n - k 1 ) - ∑ k 2 = 0 K 2  h 2  ( k 2 )  y 2  ( n - k 2 ) =  x 1  ( n ) - ∑ k 1 = 0 K 1  h 1  ( k 1 )  ( x 1  ( n - k 1 ) + s 1  ( n - k 1 ) ) -  ∑ k 2 = 0 K 2  h 2  ( k 2 )  ( x 2  ( n - k 2 ) + s 2  ( n - k 2 ) ) =  x 1  ( n ) - ∑ k 1 = 0 K 1  h 1  ( k 1 )  x 1  ( n - k 1 ) - ∑ k 2 = 0 K 2  h 2  ( k 2 )  x 2  ( n - k 2 ) -  ∑ k 1 = 0 K 1  h 1  ( k 1 )  s 1  ( n - k 1 ) - ∑ k 2 = 0 K 2  h 2  ( k 2 )  s 2  ( n - k 2 ) ( 44 )
Distortion of the desired audio signal is defined as
e x 1  ( n ) = x 1  ( n ) - ∑ k 1 = 0 K 1  h 1  ( k 1 )  x 1  ( n - k 1 ) - ∑ k 2 = 0 K 2  h 2  ( k 2 )  x 2  ( n - k 2 ) ( 45 )
and the residual noise signal is defined as
e s  ( n ) = - ∑ k 1 = 0 K 1  h 1  ( k 1 )  s 1  ( n - k 1 ) - ∑ k 2 = 0 K 2  h 2  ( k 2 )  s 2  ( n - k 2 ) ( 46 )
e(n)=e x 1 (n)+e s(n). (47)
Similar to single-channel noise suppression in the time domain, the cost function for distortion of the desired audio signal may be defined as:
E x 1 =  ∑ n  e x 1 2  ( n ) =  ∑ n  ( x 1  ( n ) - ∑ k 1 = 0 K 1  h 1  ( k 1 )  x 1  ( n - k 1 ) - ∑ k 2 = 0 K 2  h 2  ( k 2 )  x 2  ( n - k 2 ) ) 2 =  ∑ n  x 1 2  ( n ) + ∑ n  ( ∑ k 1 = 0 K 1  h 1  ( k 1 )  x 1  ( n - k 1 ) ) 2 +  ∑ n  ( ∑ k 2 = 0 K 2  h 2  ( k 2 )  x 2  ( n - k 2 ) ) 2 -  2  ∑ n  ∑ k 1 = 0 K 1  x 1  ( n )  h 1  ( k 1 )  x 1  ( n - k 1 ) -  2  ∑ n  ∑ k 2 = 0 K 2  x 1  ( n )  h 2  ( k 2 )  x 2  ( n - k 2 ) +  2  ∑ n  ∑ k 1 = 0 K 1  ∑ k 2 = 0 K 2  h 1  ( k 1 )  x 1  ( n - k 1 )  h 2  ( k 2 )  x 2  ( n - k 2 ) ( 48 )
Re-ordering of the summation yields
E x 1 = ∑ n  x 1 2  ( n ) + ∑ k 1 = 0 K 1  ∑ k 2 = 0 K 2  h 1  ( k 1 )  h 1  ( k 2 )  ∑ n  x 1  ( n - k 1 )  x 1  ( n - k 2 ) + ∑ k 1 = 0 K 1  ∑ k 2 = 0 K 2  h 2  ( k 1 )  h 2  ( k 2 )  ∑ n  x 2  ( n - k 1 )  x 2  ( n - k 2 ) - 2  ∑ k 1 = 0 K 1  h 1  ( k 1 )  ∑ n  x 1  ( n )  x 1  ( n - k 1 ) - 2  ∑ k 2 = 0 K 2  h 2  ( k 2 )  ∑ n  x 1  ( n )  x 2  ( n - k 2 ) + 2  ∑ k 1 = 0 K 1  ∑ k 2 = 0 K 2  h 1  ( k 1 )  h 2  ( k 2 )  ∑ n  x 1  ( n - k 1 )  x 2  ( n - k 2 ) . ( 49 )
r x , y  ( k ) = ∑ n  x  ( n )  y  ( n - k )   R x , y  ( k 1 , k 2 ) = ∑ n  x  ( n - k 1 )  y  ( n - k 2 )   r _ x , y = [ r x , y  ( 0 ) , r x , y  ( 1 ) , …  , r x , y  ( K ) ]   R _ _ x , y = [ R x , y  ( 0 , 0 ) R x , y  ( 0 , 1 ) ⋯ R x , y  ( 0 , K 2 ) R x , y  ( 1 , 0 ) R x , y  ( 1 , 1 ) ⋯ R x , y  ( 1 , K 2 ) ⋮ ⋮ ⋱ ⋮ R x , y  ( K 1 , 0 ) R x , y  ( K 1 , 1 ) ⋯ R x , y  ( K 1 , K 2 ) ] = [ R x , y  ( 0 , 0 ) R x , y  ( 0 , 1 ) ⋯ R x , y  ( 0 , K 2 ) R x , y  ( 1 , 0 ) R x , y  ( 0 , 0 ) ⋯ R x , y  ( 0 , K 2 - 1 ) ⋮ ⋮ ⋱ ⋮ R x , y  ( K 1 , 0 ) R x , y  ( K 1 - 1 , 0 ) ⋯ R x , y  ( 0 , 0 ) ] ( 50 )
the distortion of the desired audio signal of Equation 49 can be expressed as
E x 1 =r x 1 (0)+ h 1 T R x 1 h 1 +h 2 T R x 2 h 2−2 h 1 T r x 1 −2 h 2 T r x 1 ,x 2 +2 h 1 T R x 1 ,x 2 h 2. (51)
For ease of notation, autocorrelation is only denoted by a single signal subscript, i.e., R x=R x,x r X=r x,x and rx(k)=rx,x(k). If the desired audio source and the additive noise at the microphones are assumed to be independent, then Equation 51 can be re-written as
E x 1 = r y 1  ( 0 ) - r s 1  ( 0 ) + h _ 1 T ( R _ _ y 1 - R _ _ s 1 )  h _ 1 + h _ 2 T ( R _ _ y 2 - R _ _ s 2 )  h _ 2 - 2  h _ 1 T ( r _ y 1 - r _ s 1 ) - 2  h _ 2 T ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + 2  h _ 1 T ( R _ _ y 1 , y 2 - R _ _ s 1 , s 2 )  h _ 2 . ( 52 )
From Equation 52, the derivatives with respect to h 1 and h 2 are derived:
∂ E x 1 ∂ h _ 1 = 2  ( R _ _ y 1 - R _ _ s 1 )  h _ 1 - 2  ( r _ y 1 - r _ s 1 ) + 2  ( R _ _ y 1 , y 2 - R _ _ s 1 , s 2 )  h _ 2   ∂ E x 1 ∂ h _ 2 = 2  ( R _ _ y 2 - R _ _ s 2 )  h _ 2 - 2  ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + 2  ( R _ _ y 1 , y 2 - R _ _ s 1 , s 2 ) T  h _ 1 . ( 53 )
In a like manner to Equation 18, the cost function for the unnaturalness of the residual noise signal is initially chosen as the mean-squared error between the residual noise signal and a scaled version of the original additive noise signal:
E s 1 =  ∑ n   ( η   s 1  ( n ) - e s  ( n ) ) 2 =  ∑ n  ( η   s 1  ( n ) + ∑ k 1 = 0 K 1   h 1  ( k 1 )  s 1  ( n - k 1 ) + ∑ k 2 = 0 K 2   h 2  ( k 2 )  s 2  ( n - k 2 ) ) 2 =  ∑ n   η 2  s 1 2  ( n ) +  ∑ n  ( ∑ k 1 = 0 K 1   h 1  ( k 1 )  s 1  ( n - k 1 ) ) 2 +  ∑ n   ( ∑ k 2 = 0 K 2   h 2  ( k 2 )  s 2  ( n - k 2 ) ) 2 +  2  η  ∑ n   ∑ k 1 = 0 K 1   s 1  ( n )  h 1  ( k 1 )  s 1  ( n - k 1 ) +  2  η  ∑ n   ∑ k 2 = 0 K 2   s 1  ( n )  h 2  ( k 2 )  s 2  ( n - k 2 ) +  2  ∑ n   ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h 1  ( k 1 )  s 1  ( n - k 1 )  h 2  ( k 2 )  s 2  ( n - k 2 ) ( 54 )
Using the definitions of Equation 50, it is expressed as
E s 1 =η2 r s 1 (0)+ h 1 T R s 1 h 1 +h 2 T R s 2 h 2+2η h 1 T r s 1 +2η hh 2 T r s 1 ,s 2 +2 h 1 T R s 1 ,s 2 h 2 (55)
from which the derivatives with respect to h 1 and h 2 are derived:
∂ E s 1 ∂ h _ 1 = 2  R _ _ s 1  h _ 1 + 2  η  r _ s 1 + 2  R _ _ s 1 , s 2  h _ 2   ∂ E s 1 ∂ h _ 2 = 2  R _ _ s 2  h _ 2 + 2  η  r _ s 1 , s 2 + 2  R _ _ s 1 , s 2 T  h _ 1 . ( 56 )
Equivalently to single-channel noise suppression in the time domain, the composite cost function is constructed as a linear combination of the cost function for the distortion of the desired audio signal and the cost function for unnaturalness of the residual background noise:
E  = ⇓  α   E x 1 + ( 1 - α )  E s 1   ∂ E ∂ h _ 1 = α  ∂ E x 1 ∂ h _ 1 + ( 1 - α )  ∂ E s 1 ∂ h _ 1 = 0 _   ∂ E ∂ h _ 2 = α  ∂ E x 1 ∂ h _ 2 + ( 1 - α )  ∂ E s 1 ∂ h _ 2 = 0 _ ( 57 )
Using Equation 53 and Equation 56, the derivatives can be expanded to
∂ E ∂ h _ 1 = 2  ( α   R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 )  h _ 1 + 2  ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 )  h _ 2 - 2  α  ( r _ y 1 - r _ s 1 ) + 2  η  ( 1 - α )  r _ s 1 = 0 _   ∂ E ∂ h _ 2 = 2  ( α   R _ _ y 2 + ( 1 - 2  α )  R _ _ s 2 )  h _ 2 + 2  ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) T  h _ 1 - 2  α  ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + 2  η  ( 1 - α )  r _ s 1 , s 2 = 0 _ . ( 58 )
This can be written using the following matrix equation
[ ( α   R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 ) ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) T ( α   R _ _ y 2 + ( 1 - 2  α )  R _ _ s 2 ) ] [  h _ 1 h _ 2 ] = [  α  r _ y 1 - ( η  ( 1 - α ) + α )  r _ s 1 α  r _ y 1 , y 2 - ( η  ( 1 - α ) + α )  r _ s 1 , s 2 ] ( 59 )
and the solution for the FIR filters is given by
[ h _ 1 h _ 2 ] = [ ( α   R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 ) ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) T ( α   R _ _ y 2 + ( 1 - 2  α )  R _ _ s 2 ) ] - 1 [  α  r _ y 1 - ( η  ( 1 - α ) + α )  r _ s 1 α  r _ y 1 , y 2 - ( η  ( 1 - α ) + α )  r _ s 1 , s 2 ] ( 60 )
Comparing the solution in Equation 60 to that of the single-channel solution in Equation 21 reveals a strong resemblance between the four sub-matrices in the matrix inversion of Equation 60 and the equivalent single matrix of Equation 21. A similar resemblance is present between the right-most vectors in Equation 60 and Equation 21.
Recognizing the resemblance between Equation 60 and Equation 21 makes it easy to generalize the dual-channel solution to allow for shaping of the residual noise signal. By basically comparing the single-channel solution allowing noise shaping, Equation 33, to the solution of Equation 21 without noise shaping, the dual-channel solution is easily generalized to allow spectral shaping of the residual noise signal:
[ h _ 1 h _ 2 ] = [ ( α   R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 ) ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) ( α   R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) T ( α   R _ _ y 2 + ( 1 - 2  α )  R _ _ s 2 ) ] - 1 [  α ( r _ y 1 - r _ s 1 ) - ( 1 - α )  R _ _ s 1  h _ s α ( r _ y 1 , y 2 - r _ s 1 , s 2 ) - ( 1 - α )  R _ _ s 1  s 2 T  h _ s ] ( 61 )
Further exploiting the analogy of the single- and dual-channel solutions, the equivalent of the Wiener solution for the dual-channel noise suppression is easily deduced from Equation 60. With α=0.5 and η=0, corresponding to infinite noise attenuation, the solution is obtained as
[ h _ 1 h _ 2 ] = [ R _ _ y 1 R _ _ y 1 , y 2 ( R _ _ y 1 , y 2 ) T R _ _ y 2 ] - 1  [ r _ y 1 - r _ s 1 r _ y 1 , y 2 - r _ s 1 , s 2 ] ( 62 )
Similar to single-channel noise suppression in the time domain as previously described, in practice, the statistics of the additive noise can be estimated during segments in which the desired audio signal is absent.
An alternative formulation for deriving a time domain filter for dual-channel noise suppression will now be described. The modified analysis is performed by making similar assumptions to those described in the latter portion of Section B.1 above with respect to modifying the formulation for deriving the single-channel time domain filter. In accordance with this modified formulation, Equation 44 changes to
e  ( n ) =  x 1  ( n ) + ∑ k s = 0 K s   h s  ( k s )  s 1  ( n - k s ) - x ^ 1  ( n ) =  x 1  ( n ) + ∑ k s = 0 K s   h s  ( k s )  s 1  ( n - k s ) -  ∑ k 1 = 0 K 1   h 1  ( k 1 )  y 1  ( n - k 1 ) -  ∑ k 2 = 0 K 2   h 2  ( k 2 )  y 2  ( n - k 2 ) =  x 1  ( n ) - ∑ k 1 = 0 K 1   h 1  ( k 1 )  x 1  ( n - k 1 ) -  ∑ k 2 = 0 K 2   h 2  ( k 2 )  x 2  ( n - k 2 ) +  ∑ k s = 0 K s   h s  ( k s )  s 1  ( n - k s ) - ∑ k 1 = 0 K 1   h 1  ( k 1 )  s 1  ( n - k 1 ) -  ∑ k 2 = 0 K 2   h 2  ( k 2 )  s 2  ( n - k 2 ) ( 63 )
including the generalization to shaping of the residual noise signal. Here, the distortion of the desired audio signal is represented as
x 1  ( n ) - ∑ k 1 = 0 K 1   h 1  ( k 1 )  x 1  ( n - k 1 ) - ∑ k 2 = 0 K 2   h 2  ( k 2 )  x 2  ( n - k 2 ) ,
which is identical to Equation 45. Since the distortion of the desired audio signal remains unchanged compared to Equation 45, the derivatives of the distortion of the desired audio signal relative to the FIR filters remain unchanged. Compare Equation 52 and Equation 53:
E x 1 = r y 1  ( 0 ) - r s 1  ( 0 ) + h _ 1 T ( R _ _ y 1 - R _ _ s 1 )  h _ 1 + h _ 2 T ( R _ _ y 2 - R _ _ s 2 )  h _ 2 - 2  h _ 1 T  ( r _ y 1 - r _ s 1 ) - 2  h _ 2 T  ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + 2  h _ 1 T ( R _ _ y 1 , y 2 - R _ _ s 1 , s 2 )  h _ 2 ( 64 ) ∂ E x 1 ∂ h _ 1 = 2  ( R _ _ y 1 - R _ _ s 1 )  h _ 1 - 2  ( r _ y 1 - r _ s 1 ) + 2  ( R _ _ y 1 , y 2 - R _ _ s 1 , s 2 )  h _ 2   ∂ E x 1 ∂ h _ 2 = 2  ( R _ _ y 2 - R _ _ s 2 )  h _ 2 - 2  ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + 2  ( R _ _ y 1 , y 2 - R _ _ s 1 , s 2 ) T  h _ 1 ( 65 )
In Equation 63, the unnaturalness of the residual noise signal is given by
∑ k s = 0 K s   h s  ( k s )  s 1  ( n - k s ) - ∑ k 1 = 0 K 1   h 1  ( k 1 )  s 1  ( n - k 1 ) - ∑ k 2 = 0 K 2   h 2  ( k 2 )  s 2  ( n - k 2 ) . ( 66 )
The associated cost function is expressed as
E s 1 =  ∑ n   e s 1 2  ( n ) =  ∑ n  ( ∑ k s = 0 K s   h s  ( k s )  s 1  ( n - k s ) - ∑ k 1 = 0 K 1   h 1  ( k 1 )  s 1  ( n - k 1 ) - ∑ k 2 = 0 K 2   h 2  ( k 2 )  s 2  ( n - k 2 ) ) 2 =  ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h s  ( k 1 )  h s  ( k 2 )  ∑ n   s 1  ( n - k 1 )  s 1  ( n - k 2 ) +  ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h 1  ( k 1 )  h 1  ( k 2 )  ∑ n   s 1  ( n - k 1 )  s 1  ( n - k 2 ) +  ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h 2  ( k 1 )  h 2  ( k 2 )  ∑ n   s 2  ( n - k 1 )  s 2  ( n - k 2 ) -  2  ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h s  ( k 1 )  h 1  ( k 2 )  ∑ n   s 1  ( n - k 1 )  s 1  ( n - k 2 ) -  2  ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h s  ( k 1 )  h 2  ( k 2 )  ∑ n   s 1  ( n - k 1 )  s 2  ( n - k 2 ) +  2  ∑ k 1 = 0 K 1   ∑ k 2 = 0 K 2   h 1  ( k 1 )  h 2  ( k 2 )  ∑ n   s 1  ( n - k 1 )  s 2  ( n - k 2 ) ( 67 )
In vector and matrix notation this is expressed as
E s 1 =h s T R′h s 1 +h 1 T R s 1 h 1 +h 2 T R s 2 h 2−2 h s T R″ s 1 h 1−2 h 1−2 h s T R″ s 1 s 2 h 2+2 h 1 T R s 1 s 2 h 2 (68)
where R s 1 is a (K1+1)×(K1+1) matrix, R′s 1 is a (Ks+1)×(K2+1) matrix, R s 2 is a (K2+1)×(K2+1) matrix, R″s 1 is a (Ks+1)×(K1+1) matrix, R″s 1 s 2 is a (Ks+1)×(K2+1) matrix, and R s 1 s 2 is a (K1+1)×(K2+1) matrix. Matrices with same subscripts but different superscript have identical element values but are of different sizes. From Equation 68 the derivatives with respect to h 1 and h 2 are calculated as
∂ E s 1 ∂ h _ 1 = 2  R _ _ s 1  h _ 1 - 2  R _ _ s 1 ″   T  h _ s + 2  R _ _ s 1 , s 2  h _ 2   ∂ E s 1 ∂ h _ 2 = 2  R _ _ s 2  h 2 - 2  R _ _ s 1  s 2 ″   T  h _ s + 2  R _ _ s 1 , s 2 T  h _ 1 ( 69 )
Given the weighted overall cost function of Equation 57, the derivatives for the overall cost function are given by
∂ E ∂ h _ 1 =  α  ∂ E x 1 ∂ h _ 1 + ( 1 - α )  ∂ E s 1 ∂ h _ 1 =  2  ( α   R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 )  h _ 1 =  + 2  ( α   R _ _ y 1  y 2 + ( 1 - 2  α )  R _ _ s 1  s 2 )  h _ 2 -  2  α  ( r _ y 1 - r _ s 1 ) - 2  ( 1 - α )  R s 1 ″   T  h _ s = 0 _   ∂ E ∂ h _ 2 =  α  ∂ E x 1 ∂ h _ 2 + ( 1 - α )  ∂ E s 1 ∂ h _ 2 =  2  ( α   R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1  s 2 )  h _ 1 +  2  ( α   R _ _ y 1  y 2 + ( 1 - 2  α )  R _ _ s 1  s 2 ) T  h _ 2 -  2  α  ( r _ y 1  y 2 - r _ s 1  s 2 ) - 2  ( 1 - α )  R s 1  s 2 ″   T  h _ s = 0 _ ( 70 )
which is written in matrix form as
[ ( α  R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 ) ( α  R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) ( α  R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1  s 2 ) T ( α  R _ _ y 2 + ( 1 - 2  α )  R _ _ s 2 ) ]  [ h _ 1 h _ 2 ] = [ α  ( r _ y 1 - r _ s 1 ) + ( 1 - α )  R s 1 ″   T  h _ s α  ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + ( 1 - α )  R s 1  s 2 ″   T  h _ s ] ( 71 )
[ h _ 1 h _ 2 ] = [ ( α  R _ _ y 1 + ( 1 - 2  α )  R _ _ s 1 ) ( α  R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) ( α  R _ _ y 1 , y 2 + ( 1 - 2  α )  R _ _ s 1 , s 2 ) T ( α  R _ _ y 2 + ( 1 - 2  α )  R _ _ s 2 ) ] - 1  [ α  ( r _ y 1 - r _ s 1 ) + ( 1 - α )  R s 1 ″   T  h _ s α  ( r _ y 1 , y 2 - r _ s 1 , s 2 ) + ( 1 - α )  R s 1  s 2 ″   T  h _ s ] ( 72 )
Again, the Wiener solution is obtained as a special case with α=0.5 and h s=0. Comparing Eq. 72 to Eq. 62 reveals only a sign change on the right-most terms in the far right vector.
2. Example Dual-Channel Noise Suppressor that Uses Two Time Domain Filters
FIG. 7 is a block diagram of an example dual-channel noise suppressor 700 that uses two time domain filters in accordance with an embodiment of the present invention. Noise suppressor 700 may comprise, for example, a particular implementation of noise suppressor 602 of system 600 as described above in reference to FIG. 6. Generally speaking, noise suppressor 700 operates to receive a time domain representation of a first input audio signal that comprises a first desired audio signal and a first additive noise signal and a time domain representation of a second input audio signal that comprises a second desired audio signal and a second additive noise component. Noise suppressor 700 processes the time domain representations of the first input audio signal and the second input audio signal to produce a noise-suppressed audio signal. As shown in FIG. 7, noise suppressor 700 comprises a number of interconnected components including a statistics estimation module 702, a first parameter provider module 704, a second parameter provider module 706, a time domain filter configuration module 708, a first time domain filter 710, a second time domain filter 712, and a combiner 714.
Statistics estimation module 702 is configured to calculate estimates of statistics associated with the first input audio signal, the first additive noise signal, the second input audio signal, and the second additive noise signal for use by time domain filter configuration module 708 in configuring first time domain filter 710 and second time domain filter 712. The calculation of estimates may occur on some periodic or non-periodic basis depending upon a control scheme. In an embodiment, statistics estimation module 702 estimates statistics through correlation of the time domain representation of the first input audio signal, correlation of a time domain representation of the first additive noise signal, correlation of the time domain representation of the second input audio signal, correlation of a time domain representation of the second additive noise signal, a cross-correlation between the time domain representations of the first and second input audio signals and a cross-correlation between the time domain representations of the first and second additive noise signals. For example, statistics estimation module 702 may use auto-correlation and cross-correlation techniques to estimate the vectors r y 1 , r s 1 , r y 1 ,y 2 and r s 1 ,s 2 and the matrices R y 1 , R s 1 , R y 2 , R s 2 , R y 1 ,y 2 R s 1 ,s 2 that can be used to configure a first and second time domain filter in accordance with Equation 60.
Statistics estimation module 702 may estimate the statistics of the input audio signals and the additive noise signals across a number of segments of each of the input audio signals. A sliding window approach may be used to select the segments. Statistics estimation module 702 may update the estimated statistics each time a new segment (e.g., each time a new frame) is received for each of the two input audio signals. However, this example is not intended to be limiting, and the frequency with which the statistics are updated may vary depending upon the implementation.
Statistics estimation module 702 can estimate the statistics of the received input audio signals directly. In an embodiment in which the two input audio signals are speech signals, statistics estimation module 702 may estimate the statistics of the additive noise signals during non-speech segments, premised on the assumption that the additive noise signals will be sufficiently stationary during valid speech segments. In accordance with such an embodiment, statistics estimation module 702 may include functionality that is capable of classifying segments of the input audio signals as speech or non-speech segments. Alternatively, statistics estimation module 702 may be connected to another entity that is capable of performing such a function. Of course, numerous other methods may be used to estimate the statistics of the additive noise signals.
First parameter provider module 704 is configured to obtain a value of a parameter α that specifies a degree of balance between distortion of the first desired audio signal included in the first input audio signal and unnaturalness of a residual noise signal included in the output noise-suppressed audio signal and to provide the value of the parameter α to time domain filter configuration module 708. By way of example only, the parameter α may be that discussed above and utilized to represent the two time domain filters of Equation 60.
In one embodiment, the value of the parameter α comprises a fixed aspect of noise suppressor 700 that is determined during a design or tuning phase associated with that component. Alternatively, the value of the parameter α may be determined in response to some form of user input (e.g., responsive to user control of settings of a device that includes noise suppressor 700). In a still further embodiment, first parameter provider module 704 adaptively determines the value of the parameter α based at least in part on characteristics of the first input audio signal and/or the second input audio signal. For example, in an embodiment in which the input audio signals comprise speech signals, first parameter provider module 704 may vary the value of the parameter α such that an increased emphasis is placed on minimizing the distortion of the first desired speech signal during speech segments and such that an increased emphasis is placed on minimizing the unnaturalness of the residual noise signal during non-speech segments. Still other adaptive schemes for setting the value of parameter α may be used.
Second parameter provider module 706 is configured to obtain a value of a parameter η that specifies an amount of attenuation to be applied to the first additive noise signal included in the first input audio signal and to provide the value of the parameter η to time domain filter configuration module 708. By way of example only, the parameter η may be that discussed above and utilized to represent the two time domain filters of Equation 60.
In one embodiment, the value of the parameter η comprises a fixed aspect of noise suppressor 700 that is determined during a design or tuning phase associated with that component. Alternatively, the value of the parameter η may be determined in response to some form of user input (e.g., responsive to user control of settings of a device that includes noise suppressor 700). In a still further embodiment, second parameter provider module 706 adaptively determines the value of the parameter η based at least in part on characteristics of the first input audio signal and/or the second input audio signal.
In certain embodiments, first parameter provider module 704 determines a value of the parameter α based on a current value of the parameter η. Such an embodiment takes into account that certain values of α may provide a better trade-off between distortion of the desired audio signal and unnaturalness of the residual noise signal at different levels of noise attenuation. A scheme that derives the value of the parameter α based on the value of the parameter η may also be useful for facilitating user control of noise suppression since controlling the amount of noise attenuation may be a more intuitive and understandable operation to a user than controlling the trade-off between distortion of the first desired audio signal and unnaturalness of the residual noise signal.
Time domain filter configuration module 708 is configured to obtain estimates of statistics associated with the first and second input audio signals and the first and second additive noise signals from statistics estimation module 702, the value of the parameter α that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal provided by first parameter provider module 704, and the value of the parameter η that specifies the amount of attenuation to be applied to the first additive noise signal provided by second parameter provider module 706 and to use those values to configure first time domain filter 710 and second time domain filter 712. For example, time domain filter configuration module 708 may use these values to configure first time domain filter 710 and second time domain filter 712 in accordance with Equation 60, although this is only one example. Time domain filter configuration module 708 may re-configure first time domain filter 710 and second time domain filter 712 each time new segments of the first and second input audio signals are received or in accordance with some other periodic or non-periodic control scheme.
First time domain filter 710 is configured to filter the first input audio signal to generate a first processed audio signal. Second time domain filter 710 is configured to filter the second input audio signal to generate a second processed audio signal. The filtering operation performed by each of first time domain filter 710 and second time domain filter 712 may be controlled by at least some of the estimated statistics received from statistics estimation module 702, the value of the parameter α that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal provided by first parameter provider module 704, and the value of the parameter η that specifies the amount of attenuation to be applied to the first additive noise signal provided by second parameter provider module 706. Combiner 714 is configured to add the first processed audio signal received from first time domain filter 710 to the second processed audio signal received from second time domain filter 712 to produce the noise-suppressed audio signal. Persons skilled in the relevant art(s) will appreciate that other techniques may also be used to combine the first processed audio signal with the second processed audio signal to produce the noise-suppressed audio signal.
FIG. 8 is a block diagram of an alternate example dual-channel noise suppressor 800 that uses two time domain filters in accordance with an embodiment of the present invention. Noise suppressor 800 may also comprise, for example, a particular implementation of noise suppressor 602 of system 600 as described above in reference to FIG. 6. As shown in FIG. 8, noise suppressor 800 comprises a number of interconnected components including a statistics estimation module 802, a first parameter provider module 804, a noise shaping filter provider module 806, a time domain filter configuration module 808, a first time domain filter 810, a second time domain filter 812 and a combiner 814. Statistics estimation module 802, first parameter provider module 804, time domain filter configuration module 808, first time domain filter 810, second time domain filter 812 and combiner 814 respectively operate in essentially the same fashion as statistics estimation module 702, first parameter provider module 704, time domain filter configuration module 708, first time domain filter 710, second time domain filter 712 and combiner 714 as described above in reference to noise suppressor 700 of FIG. 7, with exceptions to be described below.
In noise suppressor 800, noise shaping filter provider module 806 is configured to provide parameters associated with a noise shaping filter h s to time domain filter configuration module 808 for use in configuring first time domain filter 810 and second time domain filter 812. For example, time domain filter configuration module 808 may utilize the parameters of the noise shaping filter noise shaping filter h s to configure first time domain filter 810 and second time domain filter 812 in accordance with Equation 61 as previously described. In contrast to noise suppressor 700 which uses a noise attenuation factor η, noise suppressor 800 allows for arbitrary shaping of the residual noise signal through provision of the noise shaping filter h s. Depending upon the implementation, the noise shaping filter h s may be specified during design or tuning of a device that includes noise suppressor 800, determined based on some form of user input, or adaptively determined based on at least characteristics associated with the first input audio signal and/or the second input audio signal.
3. Example Methods for Performing Dual-Channel Noise Suppression in the Time Domain
FIG. 9 depicts a flowchart 900 of a method for performing dual-channel noise suppression in the time domain in accordance with an embodiment of the present invention. The method of flowchart 900 may be performed, for example and without limitation, by noise suppressor 700 as described above in reference to FIG. 7 or noise suppressor 800 as described above in reference to FIG. 8. However, the method is not limited to those implementations.
As shown in FIG. 9, the method of flowchart 900 begins at step 902 in which a time domain representation of a first input audio signal is received, wherein the first input audio signal comprises a first desired audio signal and a first additive noise signal. At step 904, a time domain representation of a second input audio signal is received, wherein the second input audio signal comprises a second desired audio signal and a second additive noise signal.
At step 906, the time domain representation of the first input audio signal is passed through a first time domain filter having an impulse response that is controlled by at least a parameter that specifies a degree of balance between distortion of the first desired audio signal and unnaturalness of a residual noise signal included in a noise-suppressed audio signal. At step 908, the time domain representation of the second input audio signal is passed through a second time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal. For example, the first and second time domain filters may correspond to the two time domain filters specified by Equation 60 or 61 and the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal may comprise the parameter α included in those equations. However, these are examples only and other time domain filters may be used.
Depending upon the implementation, the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal may be determined in a variety of ways. For example, the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal may be determined based at least in part on characteristics of the first input audio signal and/or the second input audio signal.
In certain embodiments, step 906 involves passing the time domain representation of the first input audio signal through a first time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal and a noise attenuation factor and step 908 involves passing the time domain representation of the second input audio signal through a second time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal and the noise attenuation factor. For example, the first and second time domain filters may be the first and second time domain filters represented by Equation 60 and the noise attenuation factor may comprise the parameter η included in that equation. However, this is one example only and other time domain filters that include a noise attenuation factor may be used. In certain embodiments, the value of the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal is determined based on the value of the noise attenuation factor.
In other embodiments, step 906 involves passing the time domain representation of the first input audio signal through a first time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal and a noise shaping filter and step 908 involves passing the time domain representation of the second input audio signal through a second time domain filter having an impulse response that is controlled by at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal and the noise shaping filter. For example, the first and second time domain filters may be the first and second time domain filters represented by Equation 61 and the noise shaping filter may comprise the filter h s included in that equation. However, this is one example only and other time domain filters that include a noise shaping filter may be used.
In certain implementations, the method of flowchart 900 further includes estimating statistics comprising correlation of the time domain representation of the first input audio signal, correlation of a time domain representation of the first additive noise signal, correlation of the time domain representation of the second input audio signal, correlation of a time domain representation of the second additive noise signal, a cross-correlation between the time domain representation of the first input audio signal and the time domain representation of the second input audio signal, and a cross-correlation between the time domain representation of the first additive noise signal and the time domain representation of the second additive noise signal. For example and without limitation, this estimation of statistics may comprise estimating the vectors r y 1 , r s 1 , r y 1 ,y 2 and r s 1 ,s 2 and the matrices R y 1 , R s 1 , R y 2 , R s 2 , R y 1 ,y 2 R s 1 ,s 2 that can be used to configure a first and second time domain filter in accordance with Equation 60 or Equation 61.
In accordance with such an implementation, step 904 may involve passing the time domain representation of the first input audio signal through a first time domain filter having an impulse response that is a function of at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal and at least some of the estimated statistics and step 906 may involve passing the time domain representation of the second input audio signal through a second time domain filter having an impulse response that is a function of at least the parameter that specifies the degree of balance between the distortion of the first desired audio signal and the unnaturalness of the residual noise signal and at least some of the estimated statistics.
At step 910, the output of the first time domain filter is added to the output of the second time domain filter to produce the noise-suppressed audio signal. Persons skilled in the relevant art(s) will readily appreciate that techniques other than addition may be used to combine the output of the first time domain filter with the output of the second time domain filter to produce the noise-suppressed audio signal. At step 912, the noise-suppressed audio signal generated during step 910 is output. Depending upon the implementation, the noise-suppressed audio signal may then be further processed, stored, transmitted to a remote entity, or played back to a user.
D. Single-Channel Noise Suppression in the Frequency Domain in Accordance with Embodiments of the Present Invention
As noted above, FIG. 1 is a high-level block diagram of a single-channel noise suppression system 100 in accordance with an embodiment of the present invention. System 100 includes a noise suppressor 102 that applies noise suppression to a single input audio signal to generate a noise-suppressed signal, wherein the input audio signal comprises a desired audio signal and an additive noise signal. As will be discussed in more detail herein, noise suppressor 102 is configured to apply noise suppression in a manner that is controlled by at least a parameter that specifies a degree of balance between distortion of the desired audio signal and the unnaturalness of a residual noise signal included in the noise-suppressed audio signal.
In embodiments to be described in this section, noise suppressor 102 operates to receive a frequency domain representation of the input audio signal and to multiply the frequency domain representation of the input audio signal by a frequency domain gain function that is controlled at least by a parameter that specifies a degree of balance between distortion of the desired audio signal and unnaturalness of a residual noise signal included in the noise-suppressed audio signal. In the following, exemplary derivations of such a frequency domain gain function will first be described. An exemplary implementation of noise suppressor 102 that utilizes such a frequency domain gain function will then be described. Finally, exemplary methods for performing single-channel noise suppression in the frequency domain will be described.
1. Example Derivation of Frequency Domain Gain Function for Single-Channel Noise Suppression
This section derives a frequency domain variation of the single-channel time domain algorithm proposed in Section B.1. In the frequency domain the assumption of the desired audio signal and noise signal being additive results in an observed signal given by
Y(f)=X(f)+S(f), (73)
where the capital letter variables represent the discrete Fourier transform of the corresponding lower case time variables. Instead of filtering in the time domain, the noise suppression is achieved by multiplication in the frequency domain:
{circumflex over (X)}(f)=H(f)Y(f) (74)
wherein H(f) is the frequency domain noise suppression filter. As in previous sections, the target of the noise suppression may be the desired audio signal plus an attenuated (and possibly spectrally shaped) version of the original noise signal. Hence, the error of the noise suppression is defined as
E  ( f ) =  [ X  ( f ) + H s  ( f )  S  ( f ) ] - X ^  ( f ) =  [ X  ( f ) + H s  ( f )  S  ( f ) ] - H  ( f )  [ X  ( f ) + S  ( f ) ] =  X  ( f )  [ 1 - H  ( f ) ] + S  ( f )  [ H s  ( f ) - H  ( f ) ] ( 75 )
wherein Hs(f) represents the desired attenuation and possibly shaping of the residual noise signal. From Equation 75, the distortion of the desired audio signal is defined as
E x(f)=X(f)[1−H(f)] (76)
and the unnaturalness of the residual noise signal is defined as
E s(f)=S(f)[H s(f)−H(f)]. (77)
The cost function corresponding to the distortion of the desired audio signal is given by
E x =  ∑ n   e x 2  ( n ) =  1 N  ∑ f   E x  ( f )  E x *  ( f ) =  1 N  ∑ f  ( X  ( f )  [ 1 - H  ( f ) ] )  ( X  ( f )  [ 1 - H  ( f ) ] ) * =  1 N  ∑ f  ( [ Y  ( f ) - S  ( f ) ]  [ 1 - H  ( f ) ] )  ( [ Y  ( f ) - S  ( f ) ]  [ 1 - H  ( f ) ] ) * =  1 N  ∑ f  [ Y  ( f ) - S  ( f ) ]  [ Y *  ( f ) - S *  ( f ) ]  [ 1 - H  ( f ) ]  [ 1 - H *  ( f ) ] =  1 N  ∑ f  [ Y  ( f )  Y *  ( f ) + S  ( f )  S *  ( f ) - 2  Re  { Y  ( f )  S *  ( f ) } ]  [ 1 - H  ( f ) ]  [ 1 - H *  ( f ) ] =  1 N  ∑ f  [ Y  ( f )  Y *  ( f ) - S  ( f )  S *  ( f ) - 2   Re  { X  ( f )  S *  ( f ) } ]  [ 1 - H  ( f ) ]  [ 1 - H *  ( f ) ] ( 78 )
Note that with an independent desired audio signal and noise
X  ( f )  S *  ( f ) = ∑ k = - ( N - 1 ) N - 1   C XS  ( k )   - j2π   fk / N = 0   if   x  ( n )   and   s  ( n )   are   uncorrelated   and   hence   Equation   78   reduces   to ( 79 ) E x =  1 N  ∑ f  [ Y  ( f )  Y *  ( f ) - S  ( f )  S *  ( f ) ]  [ 1 - H  ( f ) ]  [ 1 - H *  ( f ) ] =  1 N  ∑ f  (  Y  ( f )  2 -  S  ( f )  2 )   1 - H  ( f )  2 ( 80 )
The cost function corresponding to the unnaturalness of the residual noise signal is given by
E s = ∑ n  e s 2  ( n ) = 1 N  ∑ f   E s  ( f )  E s *  ( f ) = 1 N  ∑ f  ( S  ( f )  [ H s  ( f ) - H  ( f ) ] )  ( S  ( f )  [ H s  ( f ) - H  ( f ) ] ) * = 1 N  ∑ f  S  ( f )  S *  ( f )  [ H s  ( f ) - H  ( f ) ]  [ H s  ( f ) - H  ( f ) ] * = 1 N  ∑ f   S  ( f )  2   H s  ( f ) - H  ( f )  2 . ( 81 )
Hence, the weighted cost function of distortion of the desired audio signal and unnaturalness of the residual noise signal, equivalently to Equation 37, is given by
E =  α   E x + ( 1 - α )  E s =  α N  ∑ f  (  Y  ( f )  2 -  S  ( f )  2 )   1 - H  ( f )  2 + 