IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007

715

Geometrically Constrained Independent
Component Analysis
Mirko Knaak, Shoko Araki, Member, IEEE, and Shoji Makino, Fellow, IEEE

Abstract—Acoustical signals are often corrupted by otherspeeches, sources, and background noise. This makes it necessary
to use some form of preprocessing so that signal processing systems
such as a speech recognizer or machine diagnosis can be effectively
employed. In this contribution, we introduce and evaluate a newalgorithm that uses independent component analysis (ICA) witha geometrical constraint [constrained ICA (CICA)]. It is basedon the fundamental similarity between an adaptive beamformer
and blind source separation with ICA, and does not suffer the
permutation problem of ICA-algorithms. Unlike conventional
ICA algorithms, CICA needs prior knowledge about the roughdirection of the target signal. However, it is more robust against an
erroneous estimation of the target direction than adaptive beamformers: CICA converges to the right solution as long as its lookdirection is closer to the target signal than to the jammer signal. A
high degree of robustness is very important since the geometricalprior of an adaptive beamformer is always roughly estimated in areverberant environment, even when the look direction is precise.
The effectiveness and robustness of the new algorithms is proventheoretically, and shown experimentally for three sources andthree microphones with several sets of real-world data.
Index Terms—Blind source separation (BSS), independent
component analysis (ICA), machine diagnosis, minimum variancebeamforming, signal enhancement, statistical signal processing.

I. INTRODUCTION
OR MANY signal processing tasks, such as speech recognition, transmission, or signal classification, the target
signal must be very well reconstructed when it is disturbed byother sources. Adaptive beamformers (ABF) and blind source
separation (BSS) are effective tools for multichannel signal
reconstruction.
Adaptive beamformers were introduced by Frost [1] and havebeen expanded by several contributers, e.g., [2], [3]. It can beshown formally that they provide optimal suppression of thejammer under certain conditions. However, as shown in thispaper they have limited robustness against erroneous parameters, particularly the delay vector in each frequency bin [4].
This is problematic since the delay vector is always roughly estimated in a reverberant environment, as shown in this paper. The
methods traditionally used to overcome this sensitivity mostlyF

Manuscript received June 7, 2004; revised January 24, 2006. The associate
editor coordinating the review of this manuscript and approving it for publication was Dr. Maurizio Omologo.
M. Knaak is with the Measurement Technology Laboratory, University ofTechnology, D-10587 Berlin, Germany and also with NTT Communication Science Laboratories, NTT Corporation, Kyoto 619-0237, Japan (e-mail: Mirko.
Knaak@iav.de).
S. Araki and S. Makino are with NTT Communication Science Laboratories,
NTT Corporation, Kyoto 619-0237, Japan.
Digital Object Identifier 10.1109/TASL.2006.876730

broaden and, thereby, flatten the directivity pattern, resulting ina trade-off between the signal suppression performance and theparameter sensitivity (e.g., [5], [6]).
Independent component analysis (ICA) is an emerging
technique for finding statistically independent components in a
multi-channel signal. The main application is BSS which has
been shown to be capable of recovering multiple sources fromtheir linear mixture if the sources are independent [7].
Due to the time-delayed superposition of the sources and reflections from the walls, the mixtures are convolutive in the fieldof acoustics. The separation of convolutive mixtures involves estimating significantly more parameters (see [8] for references)than when separating instantaneous mixtures. Frequency-domain approaches simplify the problem into instantaneous separation problems for the frequency components (e.g., [9], [10]).
The scaling and permutation ambiguities left in the recovered
frequency components become a serious problem. Different permutations of the frequency components lead to mixed outputs
and degraded separation results.
There are several approaches to overcome this problem, e.g.,[11], [12]. These algorithms are very time consuming or reduce the spectral resolution of the unmixing filter, therefore thescaling and permutation problem is still the main obstacle to themethod. Moreover, when there are more than two sources andtwo microphones, it is hard to solve the permutation problem.
Mainly due to these problems, the number of real applicationsin the acoustics field is still very limited and the separation performance is not sufficient [12].
Recent research has indicated an equivalence between ABF
and BSS, e.g., [13]. BSS is an intelligent set of ABFs with anadaptive null directivity aimed in the direction of the unnecessary sounds [14]. This equivalence suggests the application of
a geometrical constraint to ICA to solve the permutation andscaling problem.
In many applications the source position is roughly known inadvance since it can be estimated by sound localization methods(e.g., MUSIC [15]), determined by image processing or simply
known by geometry. The high sensitivity of the ABF to parameter changes however requires a very precise estimation of thesource position that cannot be normally achieved by the aforementioned methods. To make it worse, even a moderate reverberation makes a delay vector estimated by the accurate positionof the jammer source unreliable.
Geometrically constrained algorithms have been proposed by[16] and [17]. Their contribution does not assess in detail how
a rough estimation affects the performance of the algorithm. In[18] a geometrically constrained ICA algorithm (CICA) was introduced that employs the fast convergence properties of the1558-7916/$25.00 © 2006 IEEE

716

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007

FastICA [7]. In this paper, we analyze the behavior of CICA
formally, and compare its performance to other ICA and beamforming algorithms with a large set of machine sounds.
In Section II, we make basic assumptions about the mixture,and discuss the standard technologies for multichannel signal
processing, namely the ABF and ICA algorithms. Section III
analyzes possible reasons for the estimation of the delay vector
being imprecise and evaluates the consequences for the adaptive
beamformers. The new algorithm is introduced and its robustness and effectiveness are assessed theoretically in Section IV.
The experimental results are given in Section V.
II. ASSUMPTIONS ABOUT THE MIXTURESof broadband
In a settargetsources, the first source is the target sound and the others areinterfering sources. The sound is measured with an array of
microphones. The observed signals are filtered and mixed because the room acoustics impose
between each source
anda different impulse responseeach microphone

A. Blind Source Separation With ICA
Blind source separation uses ICA to estimate the unmixing
matrix . The estimation consists of two steps: sphering withand rotating with(4)
In the first step (sphering), the matrix
cipal component analysis (PCA)

is determined by prin-(5)
where are the sphered signals (the intermediary result of thesphering step).
In the second step, spatio-temporal decorrelation [19], nonlinear decorrelation [20], [21], maximization of nongaussianity[22] or nonstationary decorrelation [11] can be used to deterin (2),can bemine the rotation matrix . Similar to.
written as
B. Adaptive Beamformer
In the following, we restrict ourselves to the class of minimum variance (MV) beamformers. The optimization problem
for the MV beamformer is to minimize the energy of the output
signalIn the frequency domain, a convolutive mixture can be written, whereis a narrowband signal
ascomponent filtered fromwith a bandpass centered at . For
simplicity, the index is omitted hereafter. Small bold lettersmean vectors, capitalized bold letter represent matrices.
consists of the delay vectors, whereis thedelay vector of the target sound. Only under anechoic conditions, they can be approximated by the phase shifts caused bywith.
the time delays
is the sum of
When considering echoes and reverberation,
all echo paths. The covariance matrix of observed signals is

(1)
where
is the Hermitian (conjugate transposed) and E is theis a Hermitian matrix, it has
expectation operator. Since
nonzero eigenvalues
and orthogonal
belonging to them.
eigenvectors
The goal of the algorithms discussed here is to find an optimalof the target signal target . To achieve this goal,
estimation, or an unmixing
a coefficient vector
matrix..
.

(2)

is applied to the vector of observations as follows:

(3)

(6)
with a constraint which ensures that the energy of a signal
coming from the direction of the target is passed withoutchanges(7)
where is an arbitral constant and is an estimation and hence
is the estimated and not necessarily correct delay vector.
Using Lagrangian multipliers, the cost function is obtained as(8)Fig. 1 shows an example of a scatter plot of the observed signals
andwhen. The random vector is distributed around the mixing vectors
andwhich are addedin this figure. It is possible to draw potential coefficient vectors
in this scatter plot since they are in the same space. The coefficient vectors with the same output energy (6) are connected inthis figure.
The energy function whose contour lines are also drawn in. Hence, theFig. 1 is driven by the quadratic form
cost function has a bowl shape with one global minimum at the
point of origin, and the constraint in (7) is necessary to preventthe algorithm from going to the global (unwanted) minimum. The constraint is a line (or in the generalcase a hyperplane), drawn in Fig. 1. It has a strong impact on
the performance of the beamformer as shown in [15].

KNAAK et al.: GEOMETRICALLY CONSTRAINED INDEPENDENT COMPONENT ANALYSIS

Fig. 2. Definition of angles

Fig. 1. Scatter plots of the observed signals with the contour lines of Frost’s
cost function (6).

The optimal coefficient vector

is shown in Fig. 1

717

and . The axes are the same as those of Fig. 1.

to define the angle between two complex vectors and . In
is the inner product of the two complex valued vec(12),be the angle between the roughly estitors and . Let
mated delay vector
and the actual mixing vector
and letbe the angle between the mixing vectorsofof the jammer signal. For the visualthe target signal andization of the definition see Fig. 2. Since and are defined inthe (complex) space of demixing vectors, they are not identicalwith the spatial angles, e.g., between the locations of the sources(9)
Due to the equivalence between ABF and ICA under ideal
conditions as shown in [13], the solution in (9) is also the optimal ICA solution. An important consequence is that the vectorof the rotation matrix that reconstructs the target signal issimply the delay vector of the target signal transformed with asphering matrix (except for an amplification factor )(10)III. EFFECT OF A ROUGH ESTIMATION OF THE DELAY VECTOR
A. Reasons for an Imprecise Estimation of
A major drawback of ABFs is that they rely on the correct
estimation of the delay vector. Since the impulse response of ais estimatedroom is normally not available, the delay vector
solely from the time delays of the direct sound by
..
.

(11)

where is the distance between the microphones, is the speedof sound and is the estimated direction of arrival.
has two causes: a wrong direction
An estimation error ofof arrival , or the existence of reverberation. The latter is due tothe multiple directions of arrival while only the direct sound isused in (11) for the estimation. When reverberation is present,
the estimation becomes imprecise even when the source position
is well known.
To determine the error included in a rough estimation of ,we introduce a new measure: the delay or steering vector error
angle (SVA) . For its definition, we use a generalized cosine

(12)

(13)
Fig. 3 shows the SVA in two rooms with and without reverberation and with two levels of preciseness of the look direction.
The two levels of reverberation were achieved by using impulse
ms and a normal
responses from an anechoic room
ms . The impulse responses for the anechoic
room
room where derived by using the first portion of a measured imwas calculated by applying apulse response. The real vector
Fourier transform to the impulse response. It is clearly seen thatin the reverberant case an SVA may exist even though the lookdirection is precise [Fig. 3(c)].

B. Consequences for the Adaptive Beamformer
In order to assess the signal suppression properties of the ABF
when the estimation of is rough, this subsection evaluates the
eigenstructure of the covariance matrices used for the considered beamformers. In the derivation we restrict ourselves to the. The results are also valid in a generalcase wherecase.
describes
The output signal consists of two components;
the contribution of the target signal source and of the jammer
source to the output signal

(14)

718

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007()

1
1 = 10Fig. 3. SVA  f induced by a rough estimation and reverberation. The effect of reverberation is greater than the effect of an imprecise estimation.  is the,tms. (b) Incorrect estimation ,difference between the estimated look direction and the actual look direction. (a) Correct estimation tms. (c) Correct estimation ,tms. (d) Incorrect estimation 
,t
ms.

0

1 =0

= 128

1 = 10The eigenanalysis of the covariance matrixof the observed signals allows a further insight into these coefficients.
The covariance matrix can be written as(15)
andindicating the powers of the two sources. In
withorder to simplify the eigenanalysis, we assume that both sourcecomponents contribute the same power. This is satisfied if both
sources have similar intensities and are nearly in the same distance from the array. Then, the variances are considered to be
and the mixing vectors to have theequal. Additionally the noise is neglectedsame norm.
has the following eigenvectors
Under these assumptions,and eigenvalues:
(16)
(17)

1 =0
= 128

0

. The proof is given in the Appendix.
with
To reduce the complexity, we treat the nominator and the denominator of and separately resulting in

(18)
(19)

(20)

The complete derivation is shown in the Appendix. Equations(18)–(20) describe the influence of the jammer and the target
signal on the output signals with the angles and defined in(13). They make it possible to treat the robustness of the output
signal as a function of the roughness of the delay vector . If
is estimated correctly (or), the(and only if)

KNAAK et al.: GEOMETRICALLY CONSTRAINED INDEPENDENT COMPONENT ANALYSIS

Fig. 4. Attenuation of the target signal p (solid) and the jammer p (dashed)
as a function of . is 50 and 60 .

coefficient vector is orthogonal to the jammer signal. The target
signal is not attenuated719

Fig. 5. Convergence of the proposed constrained ICA algorithm. The algorithm
starts at the solution of the ABF and converges on the constraint line to thecorrect solution.

or similarly based on negentropy maximization with an arbitrary
with its derivation
nonlinear function(23)
Ifis roughly estimated, the ABF degrades drastically.
Fig. 4 shows the effect of a rough estimation of the delay vector, thewhen and vary. For a precise estimation
beamformer works properly: it passes the target signal withoutchanging it and minimizes the jammer signal. When the estimation error increases, the performance is reduced. For higher
values of the target signal experiences the same attenuation as
the jammer signal. The actual shape of the curves is influenced
by ; when is smaller, the decline of the attenuation starts
earlier.
IV. PROPOSED CONSTRAINED ICA
A. Derivation
To avoid the undesirable high sensitivity of the estimation ofwhich is very detracting as shown in Section III-B, we are
employing ICA to make the algorithm more robust. As mentioned above, several ICA approaches are possible. We consider
two cost functions to be applicable to the development of a constrained algorithm. The first is based on time-delayed decorrelation [10], [11] and the second is based on negentropy maximization (FastICA) [22]. As is usual in ICA approaches, sphering(5) is employed. Fig. 5 shows a scatter plot of the sphered signals. After sphering, we have the following equations based ontime-delayed decorrelation(21)
with a constraint to the target signal(22)with a constraint to the target signal(24)
where is a sphering matrix, is a rotation matrix and isa constant (see Section II-A). The function that was used in the.
experiments for this paper is
Although most ICA algorithms claim to be unconstrained
they normally employ the constraint that is unitary. It is necessary to avoid convergence to the point of origin and reducethe dimensionality of the optimization problem since it can beaccomplished on the unit hyper sphere, shown for two microphones in Fig. 5. Further, the constraint significantly simplifies
the derivation of the optimization function.
However, the constraint that the matrix is unitary implicitly
considers source signals with a unit varianceis unitaryis also unitary.

(25)

The underlying assumption of convolutive BSS is obviously that
and, therefore,. The algorithms tend to flatten
the power spectrum. Several methods have been proposed to
avoid this problem, e.g., [21].
When using ICA with the geometrical constraint of the ABF,
we have to weaken the assumption and use since the strict deployment of the constraint collides with the constraint in (22) or720

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007

Fig. 6. Cost function on the constraint. (a) Precise estimation  = 0 ,(24) as shown in Fig. 5. The column vectors need to be perpendicular, but they do not have to have a unit length. The degradedsolution is avoided by the constraint.
Unfortunately, weakening the constraint makes the derivationof the optimization rule less simple and the employment of thetime-delayed decorrelation approach impossible. Hence, only
the FastICA algorithm is taken into account in the further discussion. We investigate the deflationary algorithm proposed by
Bingham [22] as a basis. According to [23], the Kuhn–Tucker
points of (23) are(26)
when the derivation is made on the unit circle.
We can also use this condition for points ofthat are noton the unit circle. If is also a Kuhn–Tucker point on the unit
(for any ) is a Kuhn–Tucker point
hypersphere, thenof the minimization problem that constrains the solution to vectors of the same norm. According the theory of FastICA, themaximal nongaussianity only says something about the direction of the unmixing vector while the norm is not decisive. We
are looking for a vector that satisfies the constraint (24) and thathas the highest negentropy of all vectors with the same norm.
Hence, we do not change the solution of (26) by projecting it
to our constraint (24). We obtain the following algorithm, withthe convergence shown in Fig. 5= 27 . (b) Rough estimation  = 012 ,= 27 .

B. Theoretical Assessment
The new algorithm should be evaluated in terms of its convergence properties. The Newton method guarantees that the algorithms will converge to a stable point. Fig. 6(a) shows that there
is only one global minimum in the case of a precise estimationof the delay vector .
When the estimation is rough, two local minima appear[Fig. 6(b)]. The two minima belong to the two signals. Thismeans that since convergence is ensured, the algorithm converges either towards the target or the jammer. A convergence
towards the jammer signal amplifies the jammer signal and suppresses the target signal. This is equivalent to the permutation
problem of unconstrained BSS.
The shape of the minimization of the constraint and thestarting point of the iteration control the convergence. Conis closer to thevergence can be guaranteed whencorrect solution
than to the permutated solution
becauseit is then in the convergence range of . Both parameters are
influenced by the SVA in the PCA space . By using earlier
results (18) and lemma 1 in the Appendix, it can be assessed asfollows:

(27)
(28)
(30)
(29)
Equations (27)–(29) describe the iteration of our new CICA
algorithms. The index is the iteration index. The new algorithm starts with. If the estimation ofis correct, the estimation itself is already the correct solution. Then, the algorithm converges according to.
(27)–(29) to a stable point of the minimization problem

And similarly we can definein the PCA spaceas the angle between

and

(31)

KNAAK et al.: GEOMETRICALLY CONSTRAINED INDEPENDENT COMPONENT ANALYSIS

721

Fig. 7. Vectors in the space of the sphered signals. The axes are the same as
those of Fig. 5.

Convergence can be guaranteed as long as(Fig. 7)

Fig. 8. Comparison of the robustness of the CICA algorithm (solid line) and theadaptive beamformer (dashed line). SIR (signal to interference ratio) is definedas the ratio of the target and jammer signal contributions to the output.

(32)

range, the authors recommend the use of the conventional FastICA algorithm and solve the permutation with maximizing thecorrelation to the solutions in the other frequency bins.

(33)

V. REAL-WORLD ASSESSMENT

With (32), we can define the convergence range of our proposed CICA algorithm; it will converge to the target when thein (33).
misestimation is smaller than the critical angle
Otherwise it will converge to the permutated solution (jammer, where is the angle besignal). Since the critical angle is
tween
and
[see Fig. 2 and (13)], the algorithm converges
to the correct solution as long as its look direction is closer tothe target signal than to the jammer signal.
It should be noted that this condition always holds whenis not in the angle spanned byand . Since is definedby the absolute value of the scalar product, can only have apositive sign. Hence, the algorithm may also converge to thecorrect solution if is larger than the critical angle. Equation(33) is a sufficient condition for the convergence.
This condition allows us to compare the CICA algorithm with
the ABF (see Fig. 8). CICA is significantly more robust against. The separation performance ofa misestimation of
the ABF decreases continuously with a growth of the estimation
error (dashed curve), while the CICA algorithm has a constantly
good performance until the critical angle in (33).
For a real-world acoustical application, it is important to analyze (32) with realistic numbers for and in each frequency
bin. As shown earlier in Section III-A, the main error is introduced by the reverberation. A similar result can be seen here
(Fig. 9). In both cases the SVA is in almost all frequency bins
under the critical angle.
The frequency range in the reverberant case needs to be addressed separately. The angle , and thereby the critical angle, are very small in the low frequency range. Even a small
estimation error causes a wrong permutation. In that frequency

We compared the new CICA algorithm with the ICA algorithms, Infomax (INFO), and the minimum variance adaptive
beamformer (MVB) by using several tests. We measured impulse responses in a real room, and made mixtures by convolving these with the sources. The real room setup is depicted
in Fig. 10. Its reverberation time is 150 ms. Twoor threeobservation channels (microphones)
and sources were used.
As the signals of interest, we utilized the sound probes of electrical motors. The interfering sound here was the typical background noises of an assembly hall (like other motors, sewing anddrilling noise, etc.). When the number of used microphones was
two, a sound probe was interfered by one interferer, and when
the case of three microphones, a sound probe was interfered by
two interference sources. Investigations were carried out for ten
probes: Four probes are of fault-free motors: six probes consisted an unusual sound caused by a fault (commutator fault).
Such a setup is common in acoustical machine diagnosis withthe objective to assess the state of machine by its sound. In
an earlier publication the algorithm has also been tested withJapanese speech signals successfully [18].
First the algorithms are evaluated by the improvement in thesignal to interference ratio (SIR). The SIR is defined accordingwithto [20] and [24] astargettheportionof
thetargetandtargettargetthe portion of the jammer in the output signal. Since this measure is based on the energy portions contributed by the different
sources, it does not say anything about a potential distortionof the target signal. Therefore, a second measure is employedwhich measures how well the sound probes from the fault-free
motors can be distinguished from the faulty ones. Two classes

722IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007

Fig. 9. Comparing the SVA (solid line) for a real room with the critical angle (dotted line) of this mixture. (a) Without reverberation, twith reverberation t
ms, 
.

= 150

1 = 20 0 ms, 1 = 20 ; (b)Fig. 10. Diagram of the real room and the setup of the microphones and loudspeakers with (a) two and (b) three sources and observation channels. The reverberation time is t
ms.

= 150

can be separated in a certain feature space (typical features ofelectrical motors are loudness, roughness, energy at some frequencies, etc.) if the distance between the centers of the twoclasses is larger than the standard deviation of each class in thefeature space. Since the Mahalanobis distance is defined as the
distance between these centers normalized by the standard deviation it is a good measure of the distinctness of the classes andit must be greater than one to classify the classes automatically.
In the following, the Mahalanobis distance is calculated for theoriginal, disturbed, and reconstructed signals [25].
When two microphones are used, the sound probes are interfered with one interferer. Infomax does not require the direction of the motor sound, while MVB and CICA need this information. However, the ICA algorithms face the permutation
problem, what MVB and CICA do not. Whenthe permutation problem in Infomax is solved by estimating
the directivity patterns as proposed in [26]. Of course, CICA
and MVB do not need to solve the permutation problem. To assess the robustness of CICA and MVB, the look direction cor, ‘g’)
responds to the direction of the motor sound (in the first case and differs from the actual direction by 20(, ‘u’) in the second case.

In the test with measured impulse responses [Fig. 11(a) withtwo microphones and two sources], the delay vector is always
erroneous, since it is estimated by the phase shifts of the direct
sound while the actual delay vector is also influenced by the reverberation. Therefore, the MVB fails completely in this environment. The blind algorithms and CICA improve the SIR sufficiently. The same result can be seen with the Mahalanobis classifier (Table I): only signals reconstructed with the ICA-basedalgorithms have a Mahalanobis distance larger than one thatmakes it possible to distinguish between the motor sets.
Finally, the tests with measured impulse responses wererepeated with target signals interfered with two jammer sources
where CICA was compared to the ABF.
The ABF faces the same obstacle described above. Hence,
only CICA shows good results here. Fig. 12 shows the SIR
improvement for all the motors.

VI. CONCLUSION
This paper introduced and assessed a new geometrically constrained ICA algorithm. It showed the algorithm’s effectivenessKNAAK et al.: GEOMETRICALLY CONSTRAINED INDEPENDENT COMPONENT ANALYSISt

N M

Fig. 11. Improvement in the SIR for== 2 with measured impulse
responses ( = 150 ms). (a) For all motors, (b) average SIR improvement.

N M

tFig. 12. Improvement in the SIR for== 3 with measured impulse
responses ( = 150 ms) for all motors (a), average SIR improvement (b).

of complex valued vectors is not commutative, in opposite thereal valued vectors:, whereis the conjugate. An eigenvector is definedcomplex. And, therefore
as follows:

TABLE IMAHALANOBIS DISTANCES

and robustness theoretically and experimentally. The new algorithm solves the permutation problem of BSS of acoustic mixtures, particularly when the number of sources and microphones
is greater than two. Although prior knowledge is needed, the information does not need to be precise.
APPENDIXPROOFS
Lemma 1: The covariance matrix
and eigenvalues

723

A comparison of coefficients yields the following equations:Let, thenhas the eigenvectors

The solution of the quadratic equation is, and hence the eigenvalues

with.
Proof: A potential eigenvector is a linear combination of
and :
. Let
and
. It is worth to mention that the inner product

is

724

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007

When

and with a normalization the

The last terms are scalar and can be cancelled

Corollary 1: As a special case, the eigenvectors and eigenvalues for a real valued covariance matrix are

Additionally, the following holds:

(34)
(35)
Lemma 2: The nominator of

Therefore,isThe same is applicable to the second addend
The nominator ofis

The denominator of both components isUsing the geometrical meaning of the scalar productbe simplified significantly

canProof: The inverse of a hermitian matrix can be written
using the eigenvalues and eigenvectors
Further simplification can be made using the sine addition formula(36)
To reduce the complexity, we treat the nominator ofandand the denominator separately, starting with the nominatorof

The first addend is with the eigenvectors of lemma 1

It is

KNAAK et al.: GEOMETRICALLY CONSTRAINED INDEPENDENT COMPONENT ANALYSIS

Similarly, for the nominator of

Finally, for the denominator

ACKNOWLEDGMENT
The authors would like to thank S. Winter for revising thealgorithms, Dr. H. Sawada and R. Mukai for daily cooperationand valuable discussions, and Dr. D. Filbert and Dr. S. Katagiri
for their continuous encouragement.

725[8] K. Torkkola, “Blind separation of delayed and convolved mixtures,” in
Unsupervised Adaptive Filtering, S. Haykin, Ed. New York: Wiley,
2000, vol. 1.
[9] P. Smaragdis, “Blind separation of convolved mixtures in thefreuquency domain,” Neurocomput., vol. 22, pp. 21–34, 1998.
[10] C. Mejuto and J. Prinicpe, “A second-order method for blind source
separation of convolutive mixtures,” in Proc. Int. Workshop of ICA,
Aussois, France, Jan. 1999, pp. 395–400.
[11] L. Parra and C. Spence, “Convolutive blind separation of nonstationary
sources,” IEEE Trans. Speech Audio Process., vol. 8, pp. 320–327,
2000.
[12] M. Z. Ikram and D. R. Morgan, “Exploring permutation inconsistency
in blind separation of speech signals in a reverberant environment,” inProc. ICASSP, Jun. 2000, pp. 1041–1044.
[13] S. Araki, S. Makino, Y. Hinamoto, R. Mukai, T. Nishikawa, and H.
Saruwatari, “Equivalence between frequency domain blind source separation and frequency domain adaptive beamforming for convolutive
mixtures,” EURASIP J. Appl. Signal Process., no. 11, pp. 1157–1166,
2003.
[14] H. Saruwatari, S. Kurita, and K. Takeda, “Blind source separation combining frequency-domain ICA and beamforming,” in Proc. ICASSP,
Salt Lake City, UT, May 2001, pp. 2733–2736.
[15] D. Johnson and D. Dudgeon, Array Signal Processing: Concepts andTechniques. Englewood Cliffs, NJ: Prentice-Hall, 1993.
[16] M. Knaak and D. Filbert, “Acoustical semi-blind source separation formachine monitoring,” in Proc. 3rd. Int. Conf. ICA, San Diego, CA, Dec.
2001, pp. 361–366.
[17] L. Parra and C. Alvino, “Geometric source separation: Merging convolutive source separation with geometric beamforming,” in Proc. IEEE
Workshop on NNSP, 2001, pp. 273–282.
[18] M. Knaak, S. Araki, and S. Makino, “Geometrically constrained ICA
for convolutive mixtures of sound,” in Proc. ICASSP, Hong Kong, Apr.
2003, pp. 725–728.
[19] F. Asano, Y. Motomura, H. Asoh, and T. Matsui, “Effect of PCA
filter in blind source separation,” in Proc. 2nd. Int. Workshop of ICA,
Helsinki, Finland, Jul. 2000, pp. 57–62.
[20] H. Sawada, R. Mukai, S. Araki, and S. Makino, “Polar coordinate based
nonlinear function for frequency domain blind source separation,” inProc. ICASSP, Orlando, FL, May 2002, pp. 1001–1004.
[21] S. Ikeda and N. Murata, “A method of ICA in time-frequency domain,”in Int. Workshop ICA, Aussois, France, Jan. 1999, pp. 365–370.
[22] E. Bingham and A. Hyvärinen, “A fast fixed point algorithm for independent component analysis of complex valued signals,” Int. J. Neural
Networks, vol. 10, no. 1, pp. 1–8, 2000.
[23] A. Hyvärinen, “Fast and roubst fixed-point algorithms for independent
component analysis,” IEEE Trans. Neural Netw., vol. 10, no. 3, pp.
626–634, May 1999.
[24] R. Mukai, S. Araki, and S. Makino, “Separation and dereverberation
performance of frequency domain blind source separation,” in Proc.
3rd. Int. Conf. ICA, San Diego, CA, Dec. 2001, pp. 230–235.
[25] M. Knaak, M. Kunter, and D. Filbert, “Blind source separation for
acoustical machine diagnosis,” in Proc. 4th IEEE Int. Conf. Digital
Signal Processing, Santorini, Greece, Jun. 2002, pp. 401–404.
[26] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust approachto the permutation problem of frequency-domain blind source separation,” in Proc. ICASSP, Hong Kong, 2003, pp. 381–384.

REFERENCES[1] O. L. Frost, “An algorithm for linearly constrained adaptive antenna
array processing,” Proc. IEEE, vol. 60, no. 8, pp. 926–935, Aug. 1972.
[2] L. J. Griffith and C. W. Jim, “An alternative approach to linearly constrained adaptive beamforming,” IEEE Trans. Antennas Propag., vol.
AP-30, no. 1, pp. 27–34, Jan. 1982.
[3] K. M. Buckley and L. J. Griffith, “An adaptive generalized sidelobe
canceller with derivative constraints,” IEEE Trans. Antennas Propag.,vol. AP-34, no. 3, pp. 311–319, Mar. 1986.
[4] J. Bitzer, K. U. Simmer, and K.-D. Kammeyer, “Theoretical noise reduction limits of the generalized sidelobe canceller (GSC) for speech
signals,” in Proc. ICASSP, 1999, vol. 5, pp. 2965–2968.
[5] I. Thng, A. Cantoni, and Y. H. Leung, “Constraints for maximally flat
optimum broadband antenna arrays,” IEEE Trans. Signal Process., vol.
43, no. 6, pp. 1334–1347, Jun. 1995.
[6] Z. Shutao and I. Thng, “Pre-steering derivative constraint for robust
broadband antenna arrays,” in Proc. ICASSP, Salt Lake City, UT, May
2001, pp. 2913–2916.
[7] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley, 2001.

Mirko Knaak received the M.E. (Dipl.-Ing.) and
Ph.D. degrees in electrical engineering from theTechnical University of Berlin, Berlin, Germany, in1997 and 2003, respectively.
His research interests include blind source separation and beamforming. During the time of theresearch for this article, he was with Measurement
Technology Lab, University of Technology Berlin,and NTT Communication Science Laboratories,
NTT Corporation, Kyoto. He is now with IAV
GmbH, Berlin.

726

IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 2, FEBRUARY 2007

Shoko Araki (M’01) received the B.E. and the M.E.
degrees in mathematical engineering and information
physics from the University of Tokyo, Tokyo, Japan,
in 1998 and 2000, respectively.
She joined NTT Communication Science Laboratories, Kyoto, Japan, in 2000. She is now with the
NTT Communication Science Laboratories. Her research interests include array signal processing, blindsource separation applied to speech signals.
Ms. Araki received the TELECOM System
Technology Award from the Telecommunications
Advancement Foundation in 2004, the Best Paper Award of the IWAENC
in 2003, and the 19th Awaya Prize from Acoustical Society of Japan (ASJ)
in 2001. She is a member of the Institute of Electronics, Information, andCommunication Engineers of Japan, and a member of the Acoustical Societyof Japan.

Shoji Makino (A’89–M’90–SM’99–F’04) received
the B.E., M.E., and Ph.D. degrees from Tohoku University, Japan, in 1979, 1981, and 1993, respectively.
He joined NTT Corporation, Kyoto, Japan, in
1981. He is now an Executive Manager at the NTT
Communication Science Laboratories. He is alsoa Guest Professor at the Hokkaido University. His
research interests include adaptive filtering technologies and realization of acoustic echo cancellation,
blind source separation of convolutive mixtures ofspeech. He is the author or co-author of more than

200 articles in journals and conference proceedings and has been responsible
for more than 150 patents.
Dr. Makino received the TELECOM System Technology Award of the TAF
in 2004, the Best Paper Award of the IWAENC in 2003, the Paper Award of the
IEICE in 2005 and 2002, the Paper Award of the ASJ in 2005 and 2002, theAchievement Award of the IEICE in 1997, and the Outstanding Technological
Development Award of the ASJ in 1995. He is a member of both the Awards
Board and the Conference Board of the IEEE SP Society. He is an Associate
Editor of the IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING and an
Associate Editor of the EURASIP Journal on Applied Signal Processing. He is amember of the Technical Committee on Audio and Electroacoustics of the IEEESP Society as well as the Technical Committee on Blind Signal Processing of
the IEEE Circuits and Systems Society. He is also a member of the International
ICA Steering Committee and the Organizing Chair of the ICA2003 in Nara. He
is the General Chair of the WASPAA2007 in Mohonk and the General Chairof the IWAENC2003 in Kyoto. He is the Chair of the Technical Committee on
Engineering Acoustics of the IEICE and the ASJ. He is a council member of theASJ, and a member of the EURASIP, and a member of the IEICE.