International Workshop on Acoustic Echo and Noise Control (IWAENC2003), Sept. 2003, Kyoto, Japan

GEOMETRICAL UNDERSTANDING OF THE PCA SUBSPACE METHOD FOR
OVERDETERMINED BLIND SOURCE SEPARATION
Stefan WinterHiroshi Sawada

Shoji Makino

NTT Communication Science Laboratories, NTT Corporation
2-4
Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237, Japanwifan,sawada,maki  @cslab.kecl.ntt.co.jp
We found that for low frequencies the PCA-based methodbehaves similarly to the geometry-based method, while forhigh frequencies the former yields better results, since itnormally removes the noise subspace more efficiently than
the geometry-based method. This provides a better understanding of the PCA-based approach.

ABSTRACTIn this paper, we discuss approaches for blind source separation where we can use more sensors than the numberof sources for a better performance. The discussion focuses mainly on reducing the dimension of mixed signals
before applying independent component analysis. We compare two previously proposed methods. The first is based
2. BSS USING MORE SENSORS THAN THE
on principal component analysis, where noise reduction is
NUMBER OF SOURCES
achieved. The second involves selecting a subset of sensors based on the fact that a low frequency prefers a wide
The general framework of overdetermined BSS is shown inspacing and a high frequency prefers a narrow spacing. We
Fig. 1.
found that the PCA-based method behaves similarly
to thePSfrag replacements
SourcesSensors Subspace Processing
ICA
Outputgeometry-based method for low frequencies in the way that 
it emphasizes the outer sensors and yields superior resultsfor high frequencies, which provides a better understandingof the former method.


1. INTRODUCTION
Blind source separation (BSS) is a technique for estimating
original source signals using only sensor observations thatare mixtures of the original signals. If source signals aremutually independent and non-Gaussian, we can employ independent component analysis (ICA) to solve a BSS problem. Although in many cases equal numbers of source signals and sensors are assumed [1], using more sensors thansource signals (overdetermined systems) often yields better
results [2–4]. Different techniques are employed to map the
mixture signal space to the reduced dimensional output signal space.
In this paper we present the results of overdetermined
BSS based on two different methods of subspace selection.
Each provides better separation results than when the number of sensors and sources is the same. The first method utilizes the principal components obtained by principal component analysis (PCA) as described in [5]. The second method
is based on geometrical selection that depends on the frequency and sensor spacing as described in [6].
We compared the two methods by undertaking experiments using real world data in a reverberant environment.

Fig. 1. General framework of overdetermined BSS
After the mixing process there is a subspace processing
stage followed by the actual ICA stage. The subspace processing stage can be subdivided into a sphering stage anda dimension reduction stage. Their order is different in thetwo methods described here.
We consider a convolutive BSS model with  sources"!$#&%('('(')%

and +*$, sensors that give mixed012!3#&%('(')'(%
signals -/.
with added noise 45.  . The,
mixing process can be described by;0!76
@AB 0CD@AFE-/.
.
8:9 < 8/=?>

4G.



(1)

where .  stands for the impulse response from source>
to sensor 1 .
The use of more sensors than the number of sources usually improves the separation result. We can exploit the performance improvement technique known from beamforming theory. When achieving the separation, we have to apply some dimension reduction in order to map the numberThe author is on leave from the Chair of Multimedia Communicationand Signal Processing, University Erlangen-Nuremberg.

231

3. SUBSPACE SELECTION
of mixed signals to the number of output signals. It appears
to be more advantageous to reduce the dimensions before
3.1. Subspace selection based on statistical properties
rather than after ICA.
We employed a frequency-domain approach to solve the
Asano et al. proposed a BSS system that utilizes PCA for
convolutive BSS problem including the subspace processselecting a subspace [5]. PCA in general gives principal
ing. First, we calculate the frequency responses of the sepcomponents that are by definition uncorrelated and is suitedarating system, then we obtain time-domain representations
to dimension reduction [1, 2]. Here PCA is based on theby applying an inverse discrete Fourier transform (DFT).


spatial correlation matrix & ! }.
H
Hw
This has an advantage in that subspace processing and ICA
They
considerroom
reflections
tobeuncorrelated
noise
is used for instantaneous mixtures, which are easier to solve 
fromthedirectsourcesignals
inapractical
senseonI!KJ 9)B%
than convolutive ones. Time-domain signals H
condition
that
the
time
shift
betweendirectandreflected'('(')%
NM
are converted into frequency-domain time-series-/Lsignal is larger than the window length used for the short
signals O PQ%BRS0!TJ UV9)PQ%BRS%('(')'BU L PQ%BRWXM by an Y -point
time DFT. By assuming uncorrelatedness, it follows that
PZ!\[]%^P)_a`%^')'('(%bP(_c
Cd#e`short time DFT, where
Y
Y
Y
the
first  principal components with the largest eigenvalP)_R( : sampling frequency; : time dependence). After subues
contain a mixture of direct source signals and noise. 
PQ%BRWg!
space processing, we have uncorrelated signals fdenotesthe number of sources. By contrast the remaining
Jih:9)PQ%BRWB%/'(')'(%/hBPQ%BRWNMj
reduced to the dimension  .
principal
components consist solely of noise.
B

m
P

6
To obtain the frequency
responses k2l(.
of the separatingThusby selecting the subspace that is spanned by thePQ%BRWc!po\P/ PQ%BRS
system, we solve an ICA problem n
,
f
first
 principal components, dimensions are effectively rePQ%BRWI!qJrF9(PQ%BRW%('('(')%(rQsIBPQ%RSNM jo\BPmandwhere nduced by removing noise while keeping the signal of interis a +tu matrix whose elements are vxlw P/ . r l BPQ%BRW
est [10].

is a frequency-domain representation of yzl
and they are
Here it is important to note that sphering takes places
made so as to be mutually independent.
beforedimension reduction, which is based on the principal
The application of ICA to convolutive mixtures in thecomponents
found by sphering and is applied in the sphered
frequency-domain implies that we have to solve the permusignal
space.
tation problem afterwards [7, 8]. The more signals we haveto permute, the more difficult it is to characterize the com3.2. Subspace selection based on geometrical knowledge
ponents of each frequency bin uniquely and relate them tothe components of adjacent frequency bins. Thus we want
A method for blind source separation has been proposed
to have as few components as possible, i.e., the least possiusing several separating subsystems whose sensor spacing
ble number of dimensions.
could be configured individually [6]. The idea is based on
If we reduce the dimensions after ICA, we face a similarthefact that low frequencies prefer a wide sensor spacing
problem to the one that arises when solving the permutation
whereashigh frequencies prefer a narrow sensor spacing.
problem, i.e., the question of how to discriminate between
This
is
due
to the resulting phase difference, which plays athe components. We usually have more information before
key
role
in
separating
signals. Therefore three sensors wereusing ICA to select an appropriate subspace (such as eigenarranged
in
a
way
that
gave two different sensor spacings!
covariance and sensor spacing) than after using
valuesusing
one
sensorasacommonsensor as shown in Fig. 2.
ICA (eigenvalues are distorted due to scaling
ambiguity).
PSfragreplacements
Therefore it is better to reduce the dimensions before emSources
Sensors
Subband Processing + ICA
Outputploying ICA.
 
We applied the complex version of FastICA proposed in+
Separate &[9] to f P/ to obtain separated signals. By using negentropy
Bandpass
maximization as a basis, the unmixing vector for each signal
is gradually improved by
v|{+}
}

f
/



w

v

v
w

f

f

~B/wv

  /EW
v

 z

w

f
f


C

  zB
v

(2)
w
f

 



Separate &Bandpass+

v

Fig. 2. Geometry-based subspace selection
until the difference between consecutive unmixing vectors
falls below a certain threshold. }Se denotes the expectation value, w the hermitian operator and ~ the complex
conjugation. f is assumed to have a zero mean and unit
variance. m  denotes the derivative of a nonlinear function    , which was here chosen as   - !eIE - with !W[]'# . v is orthonormalized with respect to alreadyexisting unmixing vectors after each step.

The frequency range of the mixed signals was divided
into lower and higher frequency ranges. According to [6]for a frequency to be adequate for a given spacing  thecondition in (3) should be fulfilled. Here  is a parameterthat governs the degree to which the phase difference exceeds  ,  denotes the sound velocity and   stands for the-th source’s direction.

232

Table 2. Experimental conditions

Table 1. Summarized comparison
PCA-based selection

Geometry-based selection

Statistical consideration

Geometrical considerations

Different subspace for eachfrequency range

Two different subspaces

First sphering, then dimension reduction

First dimension reduction,then sphering

P




X)c& G9)FCV)ea

 



c and ¡c¢£G¤¥¢§¦¨¤W©ªa«­¬®&« ¯ secondsª kHz

Direction of sources
Distance of sensors
Length of source signals
Sampling rate
Window type
Filter length
Shifting intervalFrequency range parameter
Threshold for FastICAmm

Hanning©¯Qª points
a¡© points°V¤S¡a«i©
¡²±Q³

ous speech corpus and impulse responses in the Real World
Computing Partnership (RWCP) sound scene database fromreal acoustic environments [11]. The source directions  were estimated by the MUSIC algorithm (the direction orthogonal to the linearly mounted sensor array is ´ [²µ ) [12].
The frequency ranges were calculated based on the criteria
discussed in Sec 3.2.
_¸
  `We calculated the SNR
at output ¶ as #·[0ey l_ 


¸
y²l ¹
, where y l
is a portion of yºl
that
comes
_
from a source signal  l  and y]l ¹ 0! yzl FC y l  .
To avoid any influence of the permutation problem on
the result we selected the best permutation by calculating
the SNR in each frequency bin in a similar way to that described above. The solution is ideal under the condition thatthe permutation problem is perfectly solved.
The experimental conditions are given in Table 2.

(3)

The appropriate sensor pairs were chosen for each frequency range and separately used for separation in each frequency range. Before ICA was applied to each chosen pair,the mixed signals were sphered. It is important to note thatsphering takes places after dimension reduction, which isbased on geometrical considerations and is applied in themixed signal space.
The similarities and differences between the two subspace selection methods are summarized in Table 1.
4. EXPERIMENTAL RESULTS
4.1. Sensor selection of the PCA-based approach
We examined the behavior of the PCA-based subspace selection with regard to the resulting sensor selection. Speech
signals do not always comply with the assumptions of uncorrelatedness and independence, which are made when applying PCA and ICA to them. Therefore, to assess the ideal
behavior, we used artificial signals produced by a random
generator in the frequency-domain with the desired properties instead of real speech signals. The normalized sensor
gain depending on the frequency bin and sensor position isshown in Fig. 3. We used the experimental conditions givenin the first two lines of Table 2.
We can see that the PCA-based method also emphasizes
the outer sensors with a wide spacing for low frequencies
as the geometrical considerations in [6] suggest. However,
the remaining sensor is not excluded but contributes more
the higher the frequency becomes. Different settings usedin additional experiments revealed similar behavior, particularly for low frequencies.

Normalized sensor gain

1
0.8
0.6
0.4
0.2

0
0.06
1000

0.04
0.02
Sensor position (mm):0.0 28.3 56.6

800
600
400
0

200

Frequency bin

0Fig. 3. Normalized sensor gain with PCA-based subspace
selection
Figures 4 and 5 show the results for both methods for 12
pairs of speech signals. Figure 4 reveals that both subspacemethods show a similar behavior for low frequencies independent of added noise. This confirms that the PCA-based
approach also emphasizes the wider sensor spacing in thesame way as the geometry-based method.
However, for high frequencies, while both approaches
still perform similarly if we only account for reverberation,
the PCA-based approach works better than the geometry-

4.2. Comparison of the PCA- and geometry based approaches
To compare the PCA- and geometry-based methods, we separated mixtures that we obtained by convolving impulse responses .  and pairs of speech signals   , and op>
tionally adding artificial noise 4G.  . We used speech signals from the Acoustical Society of Japan (ASJ) continu-

233

based approach if noise is added (Fig. 5). We confirmed
the superior performance with additional experiments using
different sensor spacings.

5. CONCLUSION
We have compared two subspace methods for use as preprocessing steps in overdetermined BSS. We found that for low
frequencies the PCA-based method exhibits a similar performance to the geometry-based method because it also emphasizes the outer sensors. For high frequencies the PCAbased approach performs better when exposed to noisy speech
mixtures because due to appropriate phase difference it canutilize all pairs of sensors to suppress the noise. This deepens the geometrical understanding of the PCA-based method.

Geometry−based (without noise)
PCA−based (without noise)
Geometry−based (with noise)
PCA−based (with noise)20

SNR (dB)

18

16

14

6. REFERENCES
12[1] A. Hyvaerinen,
J. Karhunen,and E. Oja,
Independent Component Analysis, John Wiley & Sons,
New York, 2000.
[2] M. Joho, H. Mathis, and R.H. Lambert, “Overdeterminedblind source separation: using more sensors than source signals in a noisy mixture,” in Proc. ICA2000, June 2000, pp.
81–86.
[3] A. Westner and J. Bove, “Blind separation of real world audio signals using overdetermined mixtures,” in Proc. ICA99,
Jan. 1999.
[4] A. Koutras, E. Dermatas, and G. Kokkinakis, “Improving
simultaneous speech recognition in real room environments
using overdetermined blind source separation,” in Proc. Eurospeech2001, Sept. 2001, pp. 1009–1012.
[5] F. Asano, Y. Motomura, H. Asoh, and T. Matsui, “Effect of
PCA filter in blind source separation,” in Proc. ICA2000,
June 2000, pp. 57–62.
[6] H. Sawada, S. Araki, R. Mukai, and S. Makino, “Blindsource separation with different sensor spacing and filter length for each frequency range,” in IEEE International Workshop on Neural Networks for Signal Processing(NNSP2002), Sept. 2002, pp. 465–474.
[7] M. Z. Ikram and D. R. Morgan, “Exploring permutation inconsistency in blind separation of speech signals in a reverberant environment,” in Proc. ICASSP 2000, June 2000, pp.
1041–1044.
[8] H. Sawada, R. Mukai, S. Araki, and S. Makino, “A robust
approach to the permutation problem of frequency-domain
blind source separation,” in Proc. ICASSP2003, 2003, submitted.
[9] E. Bingham and A. Hyvaerinen, “A fast fixed-point algorithm for independent component analysis of complex valued signals,” International Journal of Neural Systems, vol.
10, no. 1, pp. 1–8, Feb. 2000.
[10] J.-P. Nadal, E. Korutcheva, and F. Aires, “Blind source separation in the presence of weak sources,” Neural Networks,
vol. 13, no. 6, pp. 589–596, 2000.
[11] Real World Computing Partnership,“RWCP soundscene database in real acoustic environments,”
http://tosa.mri.co.jp/sounddb/indexe.htm.
[12] H.L. Van Trees, Optimum array processing, Wiley, New
York, 2002.

10
1

2

3

4

5

6

7

Sound mixture

8

9

10

11

12

Fig. 4. Comparison of PCA- and geometry-based subspace
selection for low frequency range

30

Geometry−based (without noise)
PCA−based (without noise)
Geometry−based (with noise)
PCA−based (with noise)28

SNR (dB)

26

24

22

20

18

16

14

1

2

3

4

5

6

7

8

9

10

11

12

Sound mixture

Fig. 5. Comparison of PCA- and geometry-based subspace
selection for high frequency range

4.3. Interpreting experimental results
We can explain the similar performance of the PCA- andgeometry-based methods for low frequencies by the fact that
the PCA-based method also emphasizes the outer sensors
as shown in Sec. 4.1. This normally provides the highest possible phase difference for low frequencies, which isimportant for correctly separating the mixed signals by thesubsequent ICA stage as mentioned in Sec. 3.2.
For high frequencies the PCA-based method yields better results when noise is added because it can utilize all
the sensors (Fig. 3) whereas the geometry-based method
still utilizes only two sensors. By using all the sensors the
PCA-based approach can effectively suppress the uncorrelated noise.

234