Abstract:
A computer system ( 108 ) that processes mixtures of signals, such as speech and noise sources derived from multiple simultaneous microphone recordings, in order to separate them into their underlying sources. A source separation routine ( 124 ) optimizes a filter structure by minimizing cross powers of multiple output channels while enforcing geometric constraints on the filter response. The geometric constraints ( 209, 210, 215, 217 ) enforce desired responses for given locations of the underlying sources, based on the assumption that the sources are localized in space.

Description:
FIELD OF THE INNENTION  
         [0001]    This invention pertains generally to signal processing and more specifically to a system for performing mixed signal separation using geometric information and adaptive beamforming techniques.  
         BACKGROUND OF THE INVETION  
         [0002]    Blind source separation refers to the process of separating a composite signal into its original component signals without prior knowledge of their characteristics. This process is useful in speech recognition, multipath channel identification and equalization, improvement of the signal to interference ratio (SIR) of acoustic recordings, in surveillance applications and in the operation of hearing aids.  
           [0003]    Blind source separation of broad band signals in a multipath environment remains a difficult problem which has a number of ambiguities. Increasing the number of sensors allows improved performance but leads to ambiguities in the choice of the separating filters. There are in theory multiple filters that invert the responses in a room because there are multiple projections from the space containing microphone signals into the smaller space of signal sources. These multiple filters represent remaining degrees of freedom in terms of a sensor array response.  
           [0004]    The consistent assignment of signal contributions to different source channels across different frequencies creates a frequency permutation problem. This problem is inherent to all source separation algorithms including time domain algorithms unless the algorithm simultaneously considers different frequency bands. An estimation of such polyspectral properties is particularly difficult for nonstationary signals such as speech and the resulting algorithms are computationally expensive.  
           [0005]    The basic source separation problem is described by assuming the existence of M uncorrelated, time varying source signals source signals:  
           s(t)∈R M    
           [0006]    where the sources s(t) originate from different spatial locations. A number of sensors N (where N≧M) detect time varying signals:  
           x(t)∈R N .  
           [0007]    In a multipath environment each source j couples with sensor i through a linear transfer function A ij (τ), representing the impulse response of the corresponding source to sensor path such that:  
           x   i          (   t   )       =       ∑     j   =   1     M            ∑     τ   =   0       P   -   1                A   ij          (   τ   )                s   j          (     t   -   τ     )       .                                 
 
           [0008]    This equation can be rewritten using matrix notation (denoting the convolutions by *):  
             x ( t )= A ( t )* s ( t ).  
           [0009]    After applying the discrete time Fourier transform, this equation may be rewritten as:  
             x (ω)= A (ω) s (ω).  
           [0010]    The goal of convolutive source separation is to find finite impulse response (FIR) filters W ij (τ) that invert the effect of the convolutive mixing A(τ). This is equivalent to producing  
             y (ω)= W (ω) x (ω)  
           [0011]    that correspond to the original sources s(t).  
           [0012]    Different criteria for convolutive separation have been proposed, for example as discussed by H. -L. N. Thi and C. Jutten in “BLIND SOURCE SEPARATION FOR CONVOLUTIVE MIXTURES”, published in Signal Processing, vol. 45, no. 2, pp. 209-229 (1995). A two channel example is disclosed in U.S. Pat. No. 5,208,786 entitled MULTI-CHANNEL SIGNAL SEPARATION, issued to Weinstein et al. The &#39;786 patent models each channel as a multi-input-multi-output (MIMO) linear time invariant system. The input source signals are separated and recovered by requiring that the reconstructed signals be statistically uncorrelated. However, the decorrelation condition is insufficient to uniquely solve the problem unless one assumes that the unknown channel is a 2×2 MIMO finite impulse response filter.  
           [0013]    All convolutive separation criteria can be derived from the assumption of statistical independence of the unknown signals, usually limited to pairwise independence of the source signals. Pairwise independence implies that all cross-moments can be factored, thereby creating a number of necessary conditions for the model signal sources,  
           ∀ t,n,m,τ,i≠j: E[y   i   n ( t ) y   j   m ( t +τ)]= E [y   i   n ( t )] E[y   j   m ( t +τ)].  (Equation I)  
           [0014]    Convolutive separation requires that these conditions by satisfied for multiple delays X which correspond to the delays of the filter taps of W(τ). For stationary signals higher order criteria (multiple n, m) are required. For non-stationary signals such as speech multiple t can be used and multiple decorrelation (n=m=1) is sufficient.  
           [0015]    When using an independence criterion there remains both a permutation and scaling ambiguity. In the convolutive case the scaling ambiguity applies to each frequency group or bin resulting in a convolutive ambiguity for each source signal in the time domain. Any delayed or convolved versions of independent signals remain independent. For the independent frequency domain  
             E[y   i   n (ω) y   j   m (ω)]= E[y   i   n (ω)] E[y   j   m (ω)],  
           [0016]    there is a permutation ambiguity per each frequency for all orders n and m. For each frequency the independent frequency domain is therefore also satisfied by arbitrary scaling and assignment of indices i, j to the model sources  
           W(ω) A (ω)= P (ω) S (ω),  (Equation II)  
           [0017]    where P(ω) represents an arbitrary permutation matrix and S(ω) an arbitrary diagonal scaling matrix for each frequency. This creates the problem that contributions of a given signal source may not be assigned consistently to a single model source for different frequency bins. A given model source will therefore have contributions from different actual sources. The problem is more severe with an increasing number of channels as the possible number of permutations increases.  
           [0018]    This problem has often been considered an artifact of the frequency domain formulation of the separation criteria since the separation task is decoupled into independent separation tasks per frequency bin. For n=m=1 this ambiguity also applies to the time domain independence criteria set forth in Equation I. Even for higher orders the time domain criteria does not guarantee correct permutations.  
           [0019]    Some source separation work in the past simply ignored the problem. Others have proposed a number of solutions such as continuity in the spectra of the model sources, or the fact that the different frequency bins are often co-modulated. A rigorous way of capturing these statistical properties of multiple frequency contributions are polyspectra. However, in practice it is difficult to obtain robust statistics at multiple frequencies, in particular for non-stationary signals such as speech. In addition, the algorithms that consider combinations of frequencies are inherently computationally very demanding. Smoothness constraints on the filter coefficients in the frequency domain have also been proposed, as for example in U.S. Pat. No. 6,167,417 entitled CONVOLUTIVE BLIND SOURCE SEPARATION USING A MULTIPLE DECORRELATION METHOD, issued to Parra et al. This is equivalent to constraining the length of the filter as compared to the size of the analysis window. However, this limitation on the filter size may not always be reasonable as rather long filters are required in strongly reverberant environments.  
           [0020]    In theory only N sensors are needed to separate M=N sources. In practice, however, one may want to use more microphones (N&gt;M) to improve the performance of a real system. Ignoring the permutation and scaling ambiguities, Equation II reads W(ω)A(ω)=I, where I represents the identity matrix. For a given A(w) there is a N−M dimensional linear space of solutions W(ω), indicating that there are additional degrees of freedom when shaping a beam pattern represented by the filters W(ω).  
           [0021]    In conventional geometric and adaptive beamforming information such as microphone position and source location is often utilized. Geometric assumptions can be incorporated and implemented as linear constraints to the filter coefficients. In a multiple sidelobe canceler, for example, the response of one of the channels (channel i) is kept constant, which can be expressed as w(ω)e i =constant. The elements of the row vector w(ω)∈C N  are the filter elements to be applied to each microphone, and e i  is the ith column of the identity matrix. This is similar to the normalization condition imposed on the diagonal terms of W that is conventionally applied in blind separation algorithms. Rather than constraining a channel one can also constrain the response of a beamformer for a particular orientation.  
           [0022]    If the locations and response characteristics of each microphone is known, one can compute the free field response of a set of microphones and associated beam forming filters w(ω). For a position q, the phase and magnitude response is given by  
             r (ω, q )= w (ω) d (ω, q ),  
           [0023]    where d(ω,q)∈C N  represents the phase and magnitude response of the N microphones for a source located at q. For a linear array with omnidirectional microphones and a far field source (much beyond the array aperture squared over the wavelength of interest) the microphone response depends approximately only on the angle θ=θ(q) between the source and the linear array,  
             d (ω, q )= d (ω,θ)= e   −jω(p j /c)sin(θ) ,  
           [0024]    where p i  is the position of the ith microphone on the linear array and c is the wave propagation speed.  
           [0025]    Constraining the response to a particular orientation is simply expressed by the linear constraint on w(ω) such that r(ω,θ)=w(ω)d(ω,θ)=constant. This concept is used in the linearly constrained minimum variance (LCMV) algorithm and is also the underlying idea for generalized sidelobe cancelling. In order to obtain a robust beam it has also been suggested to require a smooth response around a desired orientation. In summary, all of these conditions or a combination of them can be expressed as linear constraints on w(ω).  
           [0026]    Most adaptive beamforming algorithms consider power as their main criteria for optimization. Sometimes power is minimized such as in noise or sidelobe cancelling in order to adaptively minimize the response at the orientation of the interfering signals. Sometimes power is maximized, as in matched filter approaches, to maximize the response of interest. As a result, these algorithms often perform suboptimally when there is cross talk from other sources.  
           [0027]    In second order source separation methods rather than considering the power of an individual beam w(ω)∈C 1×N  and an individual channel y(t)∈R 1 , one can consider powers and cross powers of multiple beams W(ω)∈C M×N  and their corresponding outputs y(t)∈R M . In the frequency domain these multiple beams and outputs correspond to the cross power spectra R yy (τ,ω). Second order blind source separation of nonstationary signals minimizes cross powers across multiple times. Off diagonal elements of the matrix R yy (τ,ω) are minimized in second order separation rather than the diagonal terms as is the case in conventional adaptive beamforming. Strict one channel power criteria has a serious cross talk or leakage problem when multiple sources are simultaneously active, especially in reverberant environments.  
         SUMMARY OF THE INVENTION  
         [0028]    The present invention addresses the need to separate the source of nonstationary broad band signals in a multipath environment. Ambiguities in source separation are addressed by adding prior information such as microphone position and by adding the assumption that the sources are localized in space. By replacing the power criteria of many adaptive beamforming algorithms by a cross power criteria a plurality of geometric source separation algorithms are obtained. To address the permutation, convolution and more general degrees of freedom for the shaping of beams, geometric information is utilized. The advantages of bllind source separation and geometric beamforming are combined by minimizing the cross power spectra for multiple times t while enforcing constraints used in conventional adaptive beamforming. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0029]    [0029]FIG. 1 depicts a system for executing a software implementation of the present invention;  
         [0030]    [0030]FIG. 2 is a flow diagram of a method of the present invention; 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0031]    The present invention estimates values of W(ω) obtained by known blind source separation techniques by making geometric assumptions regarding the signal sources. The sources are assumed to be localized at least up to the spatial resolution of a given array. The invention assumes that signals originate from the same locations for the entire frequency spectrum, permitting the formulation of geometric constraints on the filter coefficients. Different geometric constraints are introduced leading to a class of Geometric Source Separation algorithms.  
         [0032]    [0032]FIG. 1 depicts a system  100  for implementing the source separation method of the present invention. The system  100  comprises a composite signal source  126  that supplies the signal that is to be separated into its component signals and a computer system  108  that executes the geometric source separation routine of the present invention. The source  126  may contain any source of composite signals, but is illustratively shown to contain a sensor array  102 , a signal processor  104  and a recorded signal source  106 . The sensor array contains one or more transducers  102 A,  102 B and  102 C such as microphones. The transducers are coupled to a signal processor  104  that performs signal digitization. A digital signal is coupled to the computer system  108  for signal separation and further processing. A recorded signal source  106  may optionally form a source of the compositie signals that require separation.  
         [0033]    The computer system  108  comprises a central processing unit (CPU)  114 , a memory  122 , support circuits  116  and an input/output (I/O) interface  120 . The computer system  108  is generally coupled through the I/O interface  120  to a display  112  and various input devices  110  such as a mouse and keyboard. The support circuits generally contain well known circuits such as cache, power supplies, clock circuits, a communications bus and the like. The memory  122  may include random access memory (RAM), read only memory (ROM), disk drive, tape drive and the like, or some combination of memory devices.  
         [0034]    The invention is implemented as the geometric source separation routine  124  that is stored in memory  122  and executed by the CPU  114  to process the signal from the signal sources  126 . As such the computer system  108  is a general purpose computer system that becomes a specific purpose computer system when executing the routine  124  of the present invention. Although a general purpose computer is illustratively shown as a platform for implementing the invention, the invention can also be implemented in hardware as an application specific integrated circuit (ASIC), a digital signal processing (D)SP) integrated circuit, or other hardware device or devices. As such, the invention may be implemented in software, hardware or a combination of software and hardware.  
         [0035]    [0035]FIG. 2 depicts a flow diagram of the geometric source separation routine  124  of the present invention. At step  200 , the composite, mixed signal  126  is input, the signal being separated into a plurality of data frames containing data samples of the input signal x(t). The routine  200  produces discrete Fourier transform (DFT) values x(Ω) for each data frame x(t), that is, one frequency domain DFT value for each window of length T samples.  
         [0036]    At step  204 , a running time estimate of R yy (t,ω) is computed from the outputs y(t). As large T are required for typical filter sizes, simultaneous diagonalization of R yy (t,ω) for multiple times t is performed so as to minimize the sum of squares of the off diagonal elements according to the equation  213 :  
           J        (   W   )       =       ∑     t   ,   w                      α        (   ω   )                       R   yy          (     t   ,   ω     )       -     diag                   (       R   yy          (     t   ,   ω     )       )              2           ,                         
 
         [0037]    with the factorization R yy (t,ω)≈W(ω)R xx (t,ω)W H (ω). ∥. . . ∥ refers to the Frobenius norm defined as ∥M∥ 2 =Tr(MM H ). The summations over t and w will range over all time and all frequency bins for which adaptation of W will occur, respectively. For faster convergence using gradient descent the total input power per frequency α(ω) is normalized,  
         α        (   ω   )       =       ∑   t                             R   xx          (     t   ,   ω     )              -   2       .                             
 
         [0038]    This criteria is minimized with respect to the filter coefficients W. The lower bound of zero is obtained only if R yy (t,ω) is diagonal.  
         [0039]    The signal sources s(t) are localized at angles θ=[θ 1  . . . θ m ] and at a sufficient distance from the sensors  102  for a far field approximation to apply. While full three dimensional source location can be advantageously used with the present invention, the specific embodiment described here will identify souce locations simply by incident angle to the microphone array. Step  208  computes the geometric source locations based on the cross power spectra produced at step  204 . Step  219  applies a selection or switch to determine which of various criteria will be next be applied to the cross power spectra calculation. For example, if the hard constraint  210   
           diag ( W (ω) D (ω,θ)=1  
         [0040]    is applied to each filter w i (ω), then the ith row vector in W(ω) is forced to have a unit response in the direction θ i . In a gradient descent algorithm constraint  210  can simply be implemented by projecting the gradient (δJ/δw i (ω) to the linear subspace of permissible solutions with a constrained gradient. Since power or cross power minimization will try to minimize the response at the interference angles, this will lead to an equivalent singularity at those frequencies. In those cases soft constraint  210  should be selected as a regularization term  211  of the form  
           J   1 (ω)=∥ diag ( W (ω) D (ω,θ)−1∥ 2 .  
         [0041]    A second, more resrictive hard the constraint  209   
           W (ω) D (ω,θ)= I    
         [0042]    may be selected a step  219 . Hard constraint  209  imposes the conditions of hard constraint  210  to each filter w i (ω) by forcing the ith row vector in W(ω) to have a unit response in the direction θ i . Further, the hard constraint  209  requires that the ith row vector have a zero response in the direction of interfering signals θ j  where i≠j.  
         [0043]    At those frequencies where the grating lobes (periodic replica of the main lobes due to limited spatial sampling) of a beam pattern cross the interfering angles, D(ω,θ) is not invertible. In those cases it is unreasonable to try to enforce constraint  209  as a hard constraint. Rather, soft constraint  212  is selected by adding the regularization term of the form  
           J   2 (ω)=( W (ω) D (ω,θ)− I∥   2 .  
         [0044]    Step  219  performs various intialization conditions on equation  213 . In all initializations the relationship w i (ω)e i =1 is required to normalize the scale during optimization. In step  215  the filter structure is initialized to correspond to a delay-sum beamformer pointing into the orientations of the individual sources (del-sum). Using the orientations θ i  the filter coefficients of the ith beam (the row vectors w i (ω) of W(ω) are initialized with algorithm  
         w i (ω)= d (ω,θ i ) H .  
         [0045]    In step  219  the equation  213  can alternatively be initialized with beams that place zeroes at all orientations of interfering sources, that is, at angles θ/θ i  for the ith beam. The initialization filters with minimum norm that satisify those conditions can be computed explicitly with a least squares approach resulting in the initialization algorithm  216 ,  
           w   i (ω)= e   i   H   [e   i   ,D (ω, θ/θ i )] + ,  
         [0046]    where  +  indicates the psuedo-inverse and [e i ,D(ω, θ/θ i )] is a matrix containing the ith column el of the unit matrix and all but the ith column of D(ω,θ).  
         [0047]    In an on line implementation of a separation algorithm the concept of introducing geometric information through an initialization is typically not feasible as the source positions in the environment may be changing dynamically. More frequently the geometric constraints are enforced at initialization and throughout the optimization process. The constraints change dynamically as the estimated locations change. The linear constraints  209  and  210  are each typically implemented as a soft constraint with a penalty term. The problem of noninvertibility is addressed by introducing a frequency dependent weighting of the penalty term. In particular the goal is to eliminate the constraints from the optimization for those frequency bands for which D(ω,θ) is not inveritble. A rather straightforward metric for invertibility is the condition number. Therefore at steps  211  and  212  the regularization term J(W) is weighted with the the inverse of the condition number of λ(ω)=cond −1 (D(ω,θ), which converges to zero when D(ω,θ) is not invertible and remains bounded otherwise, for example as 0≦λ(ω)≦1. The total cost function  218  including frequency dependent weighting of the geometric regularization term is given by  
         J        (   W   )       +     λ                     ∑   w                    λ        (   ω   )            J     2   /   1              W        (   ω   )       .                                 
 
         [0048]    In algorithm  210  the regularization term J 1  will try to maintain the response of filters i in orientation θ i . The delay-sum beamformer  215  satisfies the condition of algorithm  210  strictly. In algorithm  209  the regularization term J 2  will in addition minimize the response for the orientations of the interfering sources. The filter structure that strictly guarantees the constraints of algorithm  209  are computed with a least squares approach as the psuedo inverse of D H (ω,θ), or by including a regularization term βI for the noninvertibility problem, resulting in algorithm  219 :  
           W (ω)= D   H (ω,θ)( D (ω,θ) D   H (ω,θ)+β I ) −1.    
         [0049]    Algorithm  217  places a zero at the angles of interfering sources, while its response in other directions is not specified. The results for algorithms  215 ,  210  and  209  exhibit a main lobe in the directions of the corresponding source. For conflicting frequency bands, where a grating lobe coincides with the location of an interfering source, multiple cross power minimization cancels the main lobe for algorithm  215 , while conserving it somewhat for algorithms  210  and  209  due to the geometric penalty. Qualitatively, the results for the data independent algorithm  219  appears to capture both main lobe and zeros in the correct location. However, its performance is inferior to the data adaptive algorithms  209 ,  210 ,  215  and  217 .  
         [0050]    Any of the algorithms  209 ,  210 ,  211  and  212  can be used to optimize the signal filter  222  and minimize cross power at step  202  using a gradient descent algorithm. The angles θ i  of the multiple sources can be identified automatically using the Multiple Signal Classification (MUSIC) algorithm, which is a method of computing directions of arrival of multiple signals using arbitrarily placed antennas with arbitrary directional responses. The MUSIC algorithm also applies to measuring frequencies of multiple sinusoids comprising a sampled data time series. When used to measure frequency it is directly suited to tapped delay line implementations. The number of sources M is assumed to be known. While the examples and algorithms given in this description are directed toward numerical operations in the frequency domain, the present invention can also be implemented in the time domain.