# Decoupled Independent Vector Analysis Algorithm for Convolutive Blind Source Separation without Orthogonality Constraint on the Demixing Matrices

Authors: Pengfei Wang; Jiong Li; Hang Zhang
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

In this paper, we consider the problem of convolutive blind source separation in frequency domain and introduce a solution to the problem in an independent vector analysis (IVA) framework. IVA utilizes both the statistical independence of different sources in each frequency bin and the statistical dependence of the same source in different frequency bins. However, most of previous works impose orthogonality constraint on the rows of each separation matrix which may undermine the separation performance. In this work, we propose a nonorthogonal IVA algorithm based on decoupled relative Newton method. This proposed algorithm updates the separation matrices row by row, and unlike deflation separation algorithm, there is no separation error accumulation arising. Simulation results are provided to show the superior convergence behavior and separation performance of the proposed algorithm.


---

## Body

## 1. Introduction

Blind source separation (BSS) has been widely researched over the last decades since it is able to estimate the source signals from their mixtures (observed sensor signals) without knowing both the mixing process and the sources, such as crosstalk separation in telecommunications [1], speech enhancement [2], and biomedical signal processing [3]. From the mixing model point of view, the works on BSS can be divided into two categories, which are instantaneous mixing and convolutive mixing. For the instantaneous mixing BSS problem, the most popular and promising method to address this issue is independent component analysis (ICA) [4–6]. ICA assumes that the source signals are mutually statistically independent and at most one source is Gaussian distributed. Based on this assumption, some separation criteria are employed, such as non-Gaussianity maximization [7] and negentropy maximization [6]. However, in many real applications such as wireless communications, the instantaneous mixing model is invalid. In this case, the sources undergo propagation time delay and reverberation which results in convolutive mixing. In the recent decades, various approaches have been proposed to deal with this case and they basically fall into two categories: time domain [8, 9] and frequency domain [10–12] algorithms. Time domain methods are mainly inspired by the existing blind deconvolution methods and the solutions usually require intensive computations due to the relationship of filter coefficients with each other. The computation load can be overcome by the frequency domain methods. By applying Fourier transform to the time domain convolutive observed signals, it can be converted to multiple independent linear mixing in the frequency domain. Subsequently, the separation methods for instantaneous mixing can be performed on every single frequency bin. However, there is a common problem for instantaneous BSS methods that the estimated signals are disordered. Therefore, further postprocessing is necessary to correct the permutation disorder at each frequency bin and then the separated signals in the time domain are reconstructed properly. Extensive works [13, 14] have been performed to solve the permutation problem, but most of these permutation correction methods do not perform consistently well.Independent vector analysis (IVA), an extension of ICA, can solve the frequency domain BSS problem efficiently and normally requires no bin-wise permutation correction post processing [10–12]. IVA algorithms extend the univariate function of ICA algorithms to multivariate function as the score function. In this way, the separation for different frequency bin data is no longer separate but joint, and the permutation problem is mitigated by exploiting the dependences of frequency bins. Various algorithms employ different multivariate prior distributions to preserve the interfrequency dependences for individual sources and corresponding nonlinear score functions are derived. However, most of these algorithms impose orthogonality constraint on the rows of each separation matrix which may undermine the separation performance. In addition, the most popular algorithms to address this issue are based on gradient descent method because of its simplicity [14, 15], but they suffer from slow convergence. Several Newton-based algorithms are proposed to speed up the convergence [12, 16]. However, performance of these algorithms, as shown in this paper, may be unsatisfactory in certain scenarios.In this paper, we propose a nonorthogonal IVA algorithm based on decoupled relative Newton method. This proposed algorithm updates the separation matrices row by row, and, unlike deflation separation algorithm, there is no separation error accumulation arising. For real data, it has been shown that the decoupled IVA algorithm may converge faster than the vector gradient decent and Newton ones [17]. This paper is an extended version of the literature [17] to the complex domain. In order to avoid the occurrence of singular solutions, some approximate solutions are taken in the process of algorithm derivation, and the improved algorithm is applied to solve the problem of convolutive blind separation. Simulation results are demonstrated to confirm its superior performance.The rest of this paper is organized as follows. The problem of IVA for frequency domain blind source separation is presented in Section2. The proposed algorithm is introduced in Section 3. Finally, the performance of the proposed algorithm is evaluated in Section 4 by means of some simulations, and our conclusions are stated in Section 5.


## 2. Frequency Domain IVA

Let there beN sensors and N independent sources; the convolutedly mixture at the nth sensor is (1)x~nm=∑j=1N∑p=0P-1anjps~jm-pwhere s~jm is jth source in the time domain, anjp is the impulse response of the channel linking jth source and nth sensor, and P is the channel order. The signals are transformed to the frequency domain using the short time Fourier transform (STFT). A sliding window is used to perform discrete Fourier transform; then a time-frequency representation of the observation signal is created. Using STFT, the sensor observation vector at time block t and frequency bin k becomes(2)xtk=Akstk1≤k≤K,1≤t≤Lwhere st[k]=s1,t[k],s2,t[k],⋯,sN,t[k]T (superscript T denotes transpose operator) is a source vector and A[k]∈CN×N is the unknown nonsingular mixing matrix. The nth source component vector (SCV) can be written as sn,tT=sn,t[1],sn,t[2],⋯,sn,t[K], which is independent of all the other SCVs. Then, the probability density function of the concatenated source vector can be written as fs1,t,s2,t,⋯,sN,t=∏n=1Nfsn,t. The IVA solution finds K demixing matrices and the corresponding source signal estimations for each frequency bin, denoted as W[k] and yt[k]=W[k]xt[k] for the kth frequency bin, respectively. nth estimated source of the kth frequency bin at time block t is yn,t[k]=wn[k]Txt[k], where wn[k]T is nth row vector of W[k].


## 3. Proposed Algorithm

### 3.1. Decoupled Relative Leaning

In the decoupled relative learning, the demixing matrixW[k] is updated through repeated left multiplications with matrices in form [5] (3)Wnewk=I+uvHWoldkuntil convergence, where I is the N×N identity matrix and u and v are two N×1 vectors to be optimized. Noting that the eigenvalues of I+uvH are 1 with multiplicity N-1 and 1+vHu, we can find that (3) is a rank-1 update as long as 1+vHu≠0. Thus a nonsingular matrix can be transformed into another arbitrary nonsingular matrix within N steps in (3).It is obvious that the decoupled relative learning transforms the matrix optimization problem into a series of vector optimization problems; thus we can design the algorithm more flexibly compared with the relative (natural) gradient algorithm. We only consider one special kind of relative rank-1 update, i.e., lettingu=en, where en is a unit vector whose nth element is 1. Then, by letting u=e1,e2,⋯,eN, we can update W[k] row by row via optimizing v. In this way, as shown in the next subsection, the calculation of the Newton-based learning algorithm is simplified.


### 3.2. Proposed Algorithm

In this paper, we use the score function [18](4)JIVA≡∑n=1NHyn,t-∑k=1Klog⁡det⁡Wk-C1where Hyn,t=E-log⁡pyn,t is the entropy of yn,t and C1 is constant which is independent of demixing matrices W[k]k=1,2,⋯,K. As processed in ICA, we select a nonlinear function G:RK×1↦R1×1 to approximate the entropy of SCV; e.g., (5)Hyn,t=EGyn,tHyn,tGyn,t=yn,tHyn,t2Without loss of generality, according to (3), the optimization of the nth row of W[k] can be written as(6)Δwnk=wn,newk-wn,oldk=ΔvHWoldkTwhere Δv is a small perturbation of v at 0. Correspondingly, the change of the nth estimated source within kth frequency bin is(7)Δyn,tk=ΔwnkTxtk=ΔvHWoldkxtk=ΔvHytkNoting thatdetWnew[k]=1+ΔvHendetWold[k], the change of cost can be written as(8)ΔJ=EGyn,t+Δyn,tkek-Gyn,t-log1+ΔvHenThen, using second-order Taylor series expansion to approximateGyn,t+Δyn,t[k]ek and log1+ΔvHen, the change of cost ΔJ can be written as(9)ΔJ≈ΔvHΔvTff∗+12ΔvHΔvTD0D1D1∗D0∗ΔvΔv∗where f=EG′yn,tyn,t[k]yt[k]-en, G′yn,tyn,t[k], G′′yn,tyn,t[k], D0=EG′′yn,tyn,t[k]yn,t[k]∗yt[k]yt[k]H, and D1=EG′′yn,tyn,t[k]yn,t[k]yt[k]yt[k]+enenH are the first- and second-order derivatives of Gyn,t with respect to yn,t[k], respectively.To design a Newton algorithm, the quadratic form in (9) is required to be positive definite. However, the quadratic form may not be positive definite when yn,t[k] does not converge to the source. Therefore, we make an approximation of the quadratic form to ensure the positive definiteness. When yn,t[k] converges to the source, D0 and D1 reduce to diagonal matrices respectively. Approximate D0 and D1 as(10)D^0=diagEG′′yn,tyn,tkyn,tk∗ytk⊙ytk∗(11)D^1=diagEG′′yn,tyn,tkyn,tkytk⊙ytk+enwhere diagd generates a diagonal matrix with diagonal element vector d and ⊙ denotes element-wise multiplication of two vectors. Then, the Hessian matrix for the ith element of Δv is d0,id1,id1,i∗d0,i, where d0,i and d1,i are the ith diagonal elements of D^0 and D^1, respectively. To ensure the positive definiteness of this Hessian matrix, a modification is made:(12)d0,i=maxd0,i,1+αd1,iwhere α is a small positive number. The Newton update rule for the ith element of Δv is derived as (13)Δvi=-d0,ifi-d1,ifi∗d0,i2-d1,i2From the derivation process of the proposed algorithm, we can see that there is no orthogonality constraint on the demixing matrices and no separation error accumulation arises.


## 3.1. Decoupled Relative Leaning

In the decoupled relative learning, the demixing matrixW[k] is updated through repeated left multiplications with matrices in form [5] (3)Wnewk=I+uvHWoldkuntil convergence, where I is the N×N identity matrix and u and v are two N×1 vectors to be optimized. Noting that the eigenvalues of I+uvH are 1 with multiplicity N-1 and 1+vHu, we can find that (3) is a rank-1 update as long as 1+vHu≠0. Thus a nonsingular matrix can be transformed into another arbitrary nonsingular matrix within N steps in (3).It is obvious that the decoupled relative learning transforms the matrix optimization problem into a series of vector optimization problems; thus we can design the algorithm more flexibly compared with the relative (natural) gradient algorithm. We only consider one special kind of relative rank-1 update, i.e., lettingu=en, where en is a unit vector whose nth element is 1. Then, by letting u=e1,e2,⋯,eN, we can update W[k] row by row via optimizing v. In this way, as shown in the next subsection, the calculation of the Newton-based learning algorithm is simplified.


## 3.2. Proposed Algorithm

In this paper, we use the score function [18](4)JIVA≡∑n=1NHyn,t-∑k=1Klog⁡det⁡Wk-C1where Hyn,t=E-log⁡pyn,t is the entropy of yn,t and C1 is constant which is independent of demixing matrices W[k]k=1,2,⋯,K. As processed in ICA, we select a nonlinear function G:RK×1↦R1×1 to approximate the entropy of SCV; e.g., (5)Hyn,t=EGyn,tHyn,tGyn,t=yn,tHyn,t2Without loss of generality, according to (3), the optimization of the nth row of W[k] can be written as(6)Δwnk=wn,newk-wn,oldk=ΔvHWoldkTwhere Δv is a small perturbation of v at 0. Correspondingly, the change of the nth estimated source within kth frequency bin is(7)Δyn,tk=ΔwnkTxtk=ΔvHWoldkxtk=ΔvHytkNoting thatdetWnew[k]=1+ΔvHendetWold[k], the change of cost can be written as(8)ΔJ=EGyn,t+Δyn,tkek-Gyn,t-log1+ΔvHenThen, using second-order Taylor series expansion to approximateGyn,t+Δyn,t[k]ek and log1+ΔvHen, the change of cost ΔJ can be written as(9)ΔJ≈ΔvHΔvTff∗+12ΔvHΔvTD0D1D1∗D0∗ΔvΔv∗where f=EG′yn,tyn,t[k]yt[k]-en, G′yn,tyn,t[k], G′′yn,tyn,t[k], D0=EG′′yn,tyn,t[k]yn,t[k]∗yt[k]yt[k]H, and D1=EG′′yn,tyn,t[k]yn,t[k]yt[k]yt[k]+enenH are the first- and second-order derivatives of Gyn,t with respect to yn,t[k], respectively.To design a Newton algorithm, the quadratic form in (9) is required to be positive definite. However, the quadratic form may not be positive definite when yn,t[k] does not converge to the source. Therefore, we make an approximation of the quadratic form to ensure the positive definiteness. When yn,t[k] converges to the source, D0 and D1 reduce to diagonal matrices respectively. Approximate D0 and D1 as(10)D^0=diagEG′′yn,tyn,tkyn,tk∗ytk⊙ytk∗(11)D^1=diagEG′′yn,tyn,tkyn,tkytk⊙ytk+enwhere diagd generates a diagonal matrix with diagonal element vector d and ⊙ denotes element-wise multiplication of two vectors. Then, the Hessian matrix for the ith element of Δv is d0,id1,id1,i∗d0,i, where d0,i and d1,i are the ith diagonal elements of D^0 and D^1, respectively. To ensure the positive definiteness of this Hessian matrix, a modification is made:(12)d0,i=maxd0,i,1+αd1,iwhere α is a small positive number. The Newton update rule for the ith element of Δv is derived as (13)Δvi=-d0,ifi-d1,ifi∗d0,i2-d1,i2From the derivation process of the proposed algorithm, we can see that there is no orthogonality constraint on the demixing matrices and no separation error accumulation arises.


## 4. Simulation and Results

In the first simulation, we consider the separation of two convoluted speech signals generated by (1), and set the channel order P=3. At each mixed signal, 1024-point FFT is done to time blocks. Figure 1 shows the separation result of two convolutedly mixed speech signals using the proposed algorithm. It indicates that the proposed algorithm is able to separate convolutive mixing successfully in frequency domain.Figure 1
Separation of convoluted speech signals using the proposed algorithm.
(a) (b) (c) (d) (e) (f)In order to numerically compare the separation performance of the proposed decoupled IVA algorithm with other IVA algorithms, multiple datasets separation for complex sources are simulated. The complex mixed signal is generated by (2). In addition, the dependence elements of SCV are generated as(14)sn,t=∑i=0βMn,izn,t-iwhere Mn,i is a real-valued K×K matrix and zn,t-i is a zero-mean vector whose entities are subject to uniform distribution. In the following simulations, β=3.In this section, we compare the proposed decoupled IVA algorithm with other two algorithms, vector gradient, and Newton updates for IVA with multivariate Gaussian model [16], respectively. Performance is assessed using interference to source ratio (ISR)(15)ISR=1KNN-1∑k=1K∑m,n=1m≠nNCm,nk2where Cm,n[k] is the m,n element of C[k]=W[k]A[k]; here we assume that no permutation exists. The smaller the value of ISR, the better the separation performance, specially, ISR=0 implies ideal separation performance. In all the simulations, the results of 100 independent realizations are averaged.Figure2 shows the convergence behavior of the ISR. We observe that, among the three algorithms, the proposed decoupled relative Newton algorithm converges in the fewest iterations, and the vector gradient algorithm performs poorly compared with the other two algorithms. In addition, the decoupled relative Newton algorithm has the lowest ISR values when the algorithm converges, which implies that the decoupled relative Newton algorithm has the best separation performance.Figure 2
Mean ISR of 100 trails versus number of iterations forK=4,N=4,T=3000 using the vector gradient, Newton and proposed decoupled relative Newton optimization algorithms.Figure3 depicts the separation performance versus the datasets number. We can observe that the separation performance of the proposed algorithm is the best among the three algorithms, and the ISR value of the proposed decoupled algorithm is lower than the other comparison algorithms around 15-20dB. Furthermore, the ISR of the three algorithms increases with the number of datasets. In addition, we also consider the influence of the number of sources. It shows that the number of sources has little effect on the separation results.Figure 3
Mean ISR of 100 trails versus the number of datasetsK forN=4, 10,T=3000 using the vector gradient, Newton, and proposed decoupled relative Newton optimization algorithms.Figure4 shows the ISR of the estimated sources as a function of the number of samples per dataset, where we can find that the ISR of the proposed algorithm is the lowest among the three algorithms and the ISR of the three algorithms increases as the number of datasets increases, with the largest rate of decrease occurring for the sample length smaller than 1000. When the sample length is smaller than 1000, vector gradient and Newton optimization algorithms have a similar separation performance, while, with the increase of sample length, the influence of the number of datasets becomes more apparent.Figure 4
Mean ISR of 100 trails versus the number of samples per dataset forK=4,8,12,N=4 using the vector gradient, Newton, and proposed decoupled relative Newton optimization algorithms.


## 5. Conclusion

In this paper, we propose a new IVA algorithm for separating convolutedly mixed signals in frequency domain which is based on decoupled relative Newton approach. This algorithm decomposes the matrix optimization problem into a series of row vector optimization problems, and the separation matrices do not need to be orthogonal. Simulation results show that the proposed algorithm converges fast and its separation performance is superior to that of other two algorithms based on vector gradient and Newton methods, respectively.



---

*Source: 9868725-2018-11-29.xml*