MULTIDIMENSIONAL INDEPENDENT COMPONENT ANALYSIS

This discussion paper proposes to generalize the notion of Independent Component Analysis (ICA) to the notion of Multidimensional Independent Component Analysis (MICA). We start from the ICA or blind source separation (BSS) model and show that it can be uniquely identified provided it is properly parameterized in terms of one-dimensional subspaces. From this standpoint, the BSS/ICA model is generalized to multidimensional components. We discuss how ICA standard algorithms can be adapted to MICA decomposition. The relevance of these ideas is illustrated by a MICA decomposition of ECG signals. 1. BLIND SOURCE SEPARATION We start by considering the blind source separation (BSS) problem in the simplest model: an vector of observations is modeled as with (1) where is a vector with statistically independent components and matrix is an invertible matrix. The entries of ! #" are referred to as the ‘source signals’. For the sake of simplicity, the discussion is restricted throughout to the case of zero-mean real signals. The source separation problem may be stated as: Identify mixing matrix and/or estimate the source signals based only on observations of and assuming only i) statistical independence of the ‘sources’ and ii) linear independence of the columns of . The strength of this model is that the two independence assumptions stated above are physically plausible in several instances and are strong enough to provide some kind of identifiability (see below), thus alleviating the need of any further modeling of the source distributions or of the mixing matrix. Indeterminacies. The source separation problem as stated above is clearly undetermined: if nothing is known a priori neither about the amplitude of a particular source $ nor about the amplitude of the corresponding column of , then a scalar factor can be exchanged between $ and $ without changing the product: $ $ %'& ($*) %'&,+ -$ ) for any real &/. 0 . Also we note that the ordering of the source signals is immaterial and is nothing but a notational device. Thus, in complete ignorance of the source distributions, source signals can be recovered at best up to a permutation, scales and signs. These indeterminations are well known and have been discussed at length in the blind source separation literature. They could be crudely expressed as: ‘under the working assumptions, the mixing matrix does not exist. ’ We now discuss what quantities can be truly determined from the distribution of in the BSS model. 2. INDEPENDENT COMPONENT ANALYSIS In the literature, the terms ‘blind source separation’ and ‘independent component analysis’ [1] are often used indifferently: they refer to the same model (1) with the same assumptions, pursue the same objectives and are addressed with the same algorithms. This is a source of confusion and a ‘waste of terminology’. It is also unfortunate because the term ‘analysis’ refers to the idea of decomposition into smaller, simpler elements and very often this decomposition is into a sum of terms, calling for an additive model rather than a multiplicative as described by eq. (1). ICA as an additive model. We claim that an interesting reformulation of the basic ICA model is obtained by defining the ‘components’ as 1$32 465 7 ($8 -$ for 9;:/9< so that model (1) can be rewritten as an additve, component-based, model: = $ > ? $ (2) This (admittedly trivial) rewriting of the original model calls for a change of standpoint. While model (1) is a multiplicative model reading: the observed vector is the the product of a mixing matrix by a source vector , model (2) is an additive model reading: the observed vector is a sum of one-dimensional independent vectors @ . This point of view is closer to the classic (and closely related) Principal Component Analysis (PCA) technique. PCA provides a decomposition of a second-order vector as the sum of its projections onto the principal axis of its covariance matrix. The PCA components are geometrically orthogonal by construction and also statistically orthogonal (uncorrelated). The more ambitious ICA approach is to look for components which are not necessarily geometrically orthogonal but are statistically independent (that is ‘more than statistically orthogonal’ since independence is much stronger than mere uncorrelation). A geometric parameterization. The new view of ICA introduced above is ‘matrix-free’. In this section, we discuss how it can be parameterized and why the appropriate parameterization is uniquely determined contrarily to the matrix-based parameterization of model (1). In the component model (2), the smallest subspace containing the : th component is referred to as the ‘component (sub)space’ for the : th component. This is indeed the one-dimensional linear subspace spanned by the : th column of . The orthogonal projector onto this subspace is denoted A $ and can be obtained as A $ 2 4B5 ! $C " $ " $ ! $ D9 :E9F (3) Proceedings of ICASSP ’98. Seattle. The knowledge of the projectors A $ for : is of course sufficient for separating the components: it is easily verified that 1$D HG A $C with G A $ 2 4B5 IA $KJ = L > A L MON (4) because matrix G AP$ is the projector onto the : -th component space orthogonally to all the other components. In (4), superscript Q denotes pseudo-inversion (we could use a regular inverse here but a pseudo-inverse is required below). For the sake of consistency i.e. to get rid of the modeling in terms of mixing matrix, it is desirable to define the projector A $ explicitly in terms of the component R$ rather than as a function of $ . This can be done for instance as A $ %BS T $ T U ) + SWV X" $ $(Y if 1$ has finite variance or by a similar trick otherwise. By focusing on the spaces containing to each component rather than on the columns of , we obtain the desired result of getting rid of the indeterminations of scale and sign. In some sense, we move from an algebraic description of a mixture in terms of a ‘mixing matrix’ to a geometric description in terms of ‘component spaces’. The global ‘parameter of interest’ in model (2) is not the mixing matrix but the set Z % A @A ) of the orthogonal projection matrices onto each of the component spaces. The last step to removing indeterminacies in the ICA model is maybe only rhetorical: it is easily seen that the unordered