performance analysis of the gaussian quasi maximum likelihood approach for independent vector analysis maximum likelihood ml estimation requires precise knowledge of the underlying statistical model in quasi ml qml a presumed model is used as a substitute to the unknown true model in the context of independent vector analysis iva we consider the gaussian qml estimate qmle of the demixing matrices set and present an approximate analysis of its asymptotic separation performance in gaussian qml the sources are presumed to be gaussian with covariance matrices specified by some educated guess the resulting quasi likelihood equations of the demixing matrices take a special form recently termed an extended sequentially drilled joint congruence sedjoco transformation which is reminiscent of though essentially different from classical joint diagonalization we show that asymptotically this qmle i e the solution of the resulting extended sedjoco transformation attains perfect separation under some mild conditions regardless of the sources true distributions and or covariance matrices in addition based on the small errors assumption we present a first order perturbation analysis of the extended sedjoco solution using the resulting closed form expressions for the errors in the solution matrices we provide closed form expressions for the resulting interference to source ratios isrs for iva moreover we prove that asymptotically the isrs depend only on the sources covariances and not on their specific distributions as an immediate consequence of this result we provide an asymptotically attainable lower bound on the resulting isrs we also present empirical results corroborating our analytical derivations of three simulation experiments concerning two possible model errors inaccurate covariance matrices and sources distribution mismodeling

compact cramer rao bound expression for independent component analysis despite of the increased interest in independent component analysis ica during the past two decades a simple closed form expression of the cramer rao bound crb for the demixing matrix estimation has not been established in the open literature in the present paper we fill this gap by deriving a simple closed form expression for the crb of the demixing matrix directly from its definition a simulation study comparing ica estimators with the crb is given

separation of an instantaneous mixture of gaussian autoregressive sources by the exact maximum likelihood approach this paper deals with the problem of blind separation of an instantaneous mixture of gaussian autoregressive sources without additive noise by the exact maximum likelihood approach the maximization of the likelihood function is divided using relaxation into two suboptimization problems solved by relaxation methods as well the first one consists of the estimation of the separating matrix when the autoregressive structure of the sources is fixed the second one aims at estimating this structure when the separating matrix is fixed we show that the first problem is equivalent to the determinant maximization of the separating matrix under nonlinear constraints we prove the existence and the consistency of the maximum likelihood estimator we also give the expression of fisher s information matrix then we study by computer simulations the performance of our estimator and show the improvement of its achievements w r t both quasimaximum likelihood and second order blind identification sobi estimators

an iterative inversion approach to blind source separation we present an iterative inversion ii approach to blind source separation bss it consists of a quasi newton method for the resolution of an estimating equation obtained from the implicit inversion of a robust estimate of the mixing system the resulting learning rule includes several existing algorithms for bss as particular cases giving them a novel and unified interpretation it also provides a justification of the cardoso and laheld 1996 step size normalization the ii method is first presented for instantaneous mixtures and then extended to the problem of blind separation of convolutive mixtures finally we derive the necessary and sufficient asymptotic stability conditions for both the instantaneous and convolutive methods to converge

joint independent subspace analysis using second order statistics this paper deals with a novel generalization of classical blind source separation bss in two directions first relaxing the constraint that the latent sources must be statistically independent this generalization is well known and sometimes termed independent subspace analysis isa second jointly analyzing several isa problems where the link is due to statistical dependence among corresponding sources in different mixtures when the data are one dimensional i e multiple classical bss problems this model known as independent vector analysis iva has already been studied in this paper we combine iva with isa and term this new model joint independent subspace analysis jisa we provide full performance analysis of jisa including closed form expressions for minimal mean square error mse fisher information and cramer rao lower bound in the separation of gaussian data the derived mse applies also for non gaussian data when only second order statistics are used we generalize previously known results on iva including its ability to uniquely resolve instantaneous mixtures of real gaussian stationary data and having the same arbitrary permutation at all mixtures numerical experiments validate our theoretical results and show the gain with respect to two competing approaches that either use a finer block partition or a different norm

blind separation of instantaneous mixture of sources via an independent component analysis in this paper we introduce a procedure for separating a multivariate distribution into nearly independent components based on minimizing a criterion defined in terms of the kullback leibner distance by replacing the unknown density with a kernel estimate we derive useful forms of this criterion when only a sample from that distribution is available we also compute the gradient and hessian of our criteria for use in an iterative minimization setting this gradient to zero yields a set of separating functions similar to the ones considered in the source separation problem except that here these functions are adapted to the observed data finally some simulations are given illustrating the good performance of the method

quasi maximum likelihood mimo blind deconvolution super and sub gaussianity versus consistency in this correspondence we consider the problem of multi input multi output mimo quasi maximum likelihood qml blind deconvolution we examine two classes of estimators which are commonly believed to be suitable for super and sub gaussian sources we state the consistency conditions and demonstrate a source distribution for which the studied estimators are unsuitable in the sense that they are inconsistent

robust independent component analysis by iterative maximization of the kurtosis contrast with algebraic optimal step size independent component analysis ica aims at decomposing an observed random vector into statistically independent variables deflation based implementations such as the popular one unit fastica algorithm and its variants extract the independent components one after another a novel method for deflationary ica referred to as robustica is put forward in this paper this simple technique consists of performing exact line search optimization of the kurtosis contrast function the step size leading to the global maximum of the contrast along the search direction is found among the roots of a fourth degree polynomial this polynomial rooting can be performed algebraically and thus at low cost at each iteration among other practical benefits robustica can avoid prewhitening and deals with real and complex valued mixtures of possibly noncircular sources alike the absence of prewhitening improves asymptotic performance the algorithm is robust to local extrema and shows a very high convergence speed in terms of the computational cost required to reach a given source extraction quality particularly for short data records these features are demonstrated by a comparative numerical analysis on synthetic data robustica s capabilities in processing real world data involving noncircular complex strongly super gaussian sources are illustrated by the biomedical problem of atrial activity aa extraction in atrial fibrillation af electrocardiograms ecgs where it outperforms an alternative ica based technique

blind separation of parametric nonlinear mixtures of possibly autocorrelated and non stationary sources in this paper we present a new method formulated in a maximum likelihood framework for blindly separating nonlinear mixtures of statistically independent signals our method exploits on the one hand the knowledge of the parametric model of the mixing transformation with unknown parameter values and on the other hand the possible structure of source signals i e their autocorrelation and or nonstationarity one of the main advantages of the proposed method is that it can be implemented even if the analytical expression of the inverse model is unknown the method is first addressed in a general configuration then detailed for two special cases i e a simple bijective toy model and a linear quadratic model the study of the toy model is interesting because of its simplicity and its global bijectivity which allows us to focus our efforts on parameter estimation the linear quadratic model is chosen due to its capacity to describe real world mixing phenomena simulation results using the toy model and using a subclass of the linear quadratic model i e the bilinear model show that taking into account the nonlinearity of the mixing transformations and the structure of signals considerably improves separation performance

estimation of non normalized statistical models by score matching one often wants to estimate statistical models where the probability density function is known only up to a multiplicative normalization constant typically one then has to resort to markov chain monte carlo methods or approximations of the normalization constant here we propose that such models can be estimated by minimizing the expected squared distance between the gradient of the log density given by the model and the gradient of the log density of the observed data while the estimation of the gradient of log density function is in principle a very difficult non parametric problem we prove a surprising result that gives a simple formula for this objective function the density function of the observed data does not appear in this formula which simplifies to a sample average of a sum of some derivatives of the log density given by the model the validity of the method is demonstrated on multivariate gaussian and independent component analysis models and by estimating an overcomplete filter set for natural image data

blind separation of instantaneous mixture of sources based on order statistics in this paper we introduce a novel procedure for separating an instantaneous mixture of sources based on order statistics the method is derived in a general context of independence component analysis using a contrast function defined in term of the kullback leibler divergence or of the mutual information we introduce a discretized form of this contrast permitting its easy estimation through order statistics we show that the local contrast property is preserved and derive a global contrast exploiting only the information of the support of the distribution in case this support is finite some simulations are given illustrating the good performance of the method

deconvolution of multiple time series the problem of estimating simultaneously two convolved vector time series corrupted by additive noise is considered by regarding one of the series as being stochastic and the other as fixed it is shown that the fixed component can be estimated by maximizing a frequency domain approximation to the likelihood the stochastic series is estimated by using an approximation to the conditional mean evaluated at the current maximum likelihood estimators an example involving a multiple deconvolution of seismic source and receiver functions is given

kernel methods for measuring independence we introduce two new functionals the constrained covariance and the kernel mutual information to measure the degree of independence of random variables these quantities are both based on the covariance between functions of the random variables in reproducing kernel hilbert spaces rkhss we prove that when the rkhss are universal both functionals are zero if and only if the random variables are pairwise independent we also show that the kernel mutual information is an upper bound near independence on the parzen window estimate of the mutual information analogous results apply for two correlation based dependence functionals introduced earlier we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels and prove the latter to be an upper bound on the mutual information near independence the performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis

ica mixture models for unsupervised classification of non gaussian classes and automatic context switching in blind signal separation an unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent non gaussian densities the algorithm estimates the density of each class and is able to model class distributions with non gaussian structure the new algorithm can improve classification accuracy compared with standard gaussian mixture models when applied to blind source separation in nonstationary environments the method can switch automatically between classes which correspond to contexts with different mixing properties the algorithm can learn efficient codes for images containing both natural scenes and text this method shows promise for modeling non gaussian structure in high dimensional data and has many potential applications

relative optimization for blind deconvolution we propose a relative optimization framework for quasi maximum likelihood qml blind deconvolution and the relative newton method as its particular instance special hessian structure allows fast newton system construction and solution resulting in a fast convergent algorithm with iteration complexity comparable to that of gradient methods we also propose the use of rational infinite impulse response iir restoration kernels which constitute a richer family of filters than the traditionally used finite impulse response fir kernels we discuss different choices of nonlinear functions that are suitable for deconvolution of super and sub gaussian sources and formulate the conditions under which the qml estimation is stable simulation results demonstrate the efficiency of the proposed methods

blind separation of multi dimensional components via subspace decomposition performance analysis a prevalent approach to blind separation of multi dimensional data is a two step procedure in the first step the data are assigned a one dimensional model a separating algorithm is applied according to this model this step corresponds to classical blind source separation bss in the second step the output is assigned into groups representing the multi dimensional components in this paper we consider an even more general case in which the subpartition of the components in the first step may be into elements of any dimension not necessarily one we consider a piecewise stationary model and assume that the number and dimensions of the underlying multi dimensional components are known we obtain a closed form analytical expression for the mean square error mse in the estimation of the multi dimensional components using this two step procedure as expected this approach is suboptimal in the presence of finite data size errors therefore we can predict the expected gain from using the correct model of the components over any finer decomposition thereof in addition we demonstrate the dependence of this gain on the model parameters

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

blind separation of instantaneous mixtures of nonstationary sources most source separation algorithms are based on a model of stationary sources however it is a simple matter to take advantage of possible nonstationarities of the sources to achieve separation this paper develops novel approaches in this direction based on the principles of maximum likelihood and minimum mutual information these principles are exploited by efficient algorithms in both the off line case via a new joint diagonalization procedure and in the on line case via a newton like procedure some experiments showing the good performance of our algorithms and evidencing an interesting feature of our methods are presented their ability to achieve a kind of super efficiency the paper concludes with a discussion contrasting separating methods for non gaussian and nonstationary models and emphasizing that as a matter of fact what makes the algorithms work is strictly speaking not the nonstationarity itself but rather the property that each realization of the source signals has a time varying envelope

algorithms for complex ml ica and their stability analysis using wirtinger calculus we derive a class of algorithms for independent component analysis ica based on maximum likelihood ml estimation and perform stability analysis of natural gradient ml ica with and without the constraint for unitary demixing matrix in the process we demonstrate how wirtinger calculus facilitates derivations and most importantly performing second order analysis in the complex domain and eliminates the need for making simplifying assumptions we derive natural gradient complex ml ica update rule and its variant with a unitary constraint as well as a newton algorithm for better convergence behavior the conditions for local stability are derived and studied using a generalized gaussian density ggd source model when the sources are circular and non gaussian we show analytically that both the ml and ml unitary ica update rules converge to the inverse of mixing matrix subject to a phase shift when the sources are noncircular and non gaussian we show that the nonunitary ml ica update rule is more stable than the ml unitary ica update rule when the sources are noncircular gaussians both update rules are stable only when the sources have distinct noncircularity indices simulation results are given to support these results

blind separation of gaussian sources with general covariance structures bounds and optimal estimation we consider the separation of gaussian sources exhibiting general arbitrary not necessarily stationary covariance structures first assuming a semi blind scenario in which the sources covariance structures are known we derive the maximum likelihood estimate of the separation matrix as well as the induced cramer rao lower bound icrlb on the attainable interference to source ratio isr we then extend our results to the fully blind scenario in which the covariance structures are unknown we show that under a scaling convention the fisher information matrix in this case is block diagonal implying that the same icrlb as in the semi blind scenario applies in this case as well subsequently we demonstrate that the same semi blind optimal performance can be approached asymptotically in the fully blind scenario if the sources are sufficiently ergodic or if multiple snapshots are available

frequency domain blind deconvolution based on mutual information rate in this paper a new blind single input single output siso deconvolution method based on the minimization of the mutual information rate of the deconvolved output is proposed the method works in the frequency domain and requires estimation of the signal probability density function thus the algorithm uses higher order statistics except for gaussian source and allows non minimum phase filter estimation in practice the criterion contains a regularization term for limiting noise amplification as in wiener filtering the score function estimation which represents a key point of the algorithm is detailed and the most robust estimate is selected finally experiments point to the relevance of the proposed algorithm 1 any filter minimum phase or not can be estimated and 2 on actual data underwater explosions seismovolcanic phenomena this deconvolution algorithm provides good results with a better tradeoff between deconvolution quality and noise amplification than existing methods

a generic framework for blind source separation in structured nonlinear models this paper is concerned with blind source separation in nonlinear models special attention is paid to separability issues results show that separation is impossible in the general case however for specific nonlinear models the problem becomes tractable a generic set of parametric nonlinear mixtures is considered this set has the lie group structure a group structure with continuous binary operation in the parameter set a definition of a relative gradient is given and is used to design both batch and stochastic algorithms for the latter it is shown how a proper use of the relative gradient leads to equivariant adaptive algorithms

thermodynamics of inequalities from precariousness to economic stratification growing economic inequalities are observed in several countries throughout the world following pareto the power law structure of these inequalities has been the subject of much theoretical and empirical work but their nonequilibrium dynamics e g after a policy change remains incompletely understood here we introduce a thermodynamical theory of inequalities based on the analogy between economic stratification and statistical entropy within this framework we identify the combination of upward mobility with precariousness as a fundamental driver of inequality we formalize this statement by a second law inequality displaying upward mobility and precariousness as thermodynamic conjugate variables we estimate the time scale for the relaxation of the wealth distribution after a sudden change of the after tax return on capital our method can be generalized to gain insight into the dynamics of inequalities in any markovian model of socioeconomic interactions

a direct solution for blind separation of sources proposes a direct estimation of the mixing matrix in the source separation problem as a new result we prove that for non gaussian sources the mixing matrix can be obtained by solving a fourth degree polynomial equation that derives from nonlinear equations only involving fourth order cumulants

a sequentially drilled joint congruence sedjoco transformation with applications in blind source separation and multiuser mimo systems we consider a particular form of the classical approximate joint diagonalization ajd problem which we call a sequentially drilled joint congruence sedjoco transformation the problem consists of a set of symmetric real valued or hermitian symmetric complex valued target matrices the number of matrices in the set equals their dimension and the joint diagonality criterion requires that in each transformed diagonalized target matrix all off diagonal elements on one specific row and column corresponding to the matrix index in the set be exactly zeros yet does not care about the other diagonal or off diagonal elements the motivation for this form arises in at least two different contexts maximum likelihood blind or semiblind source separation and coordinated beamforming for multiple input multiple output mimo broadcast channels we prove that sedjoco always has a solution when the target matrices are positive definite we also propose two possible iterative solution algorithms based on defining and optimizing two different criteria functions using newton s method for the first function and successive jacobi like transformations for the second the algorithms convergence behavior and the attainable performance in the two contexts above are demonstrated in simulation experiments

blind deconvolution of images using optimal sparse representations the relative newton algorithm previously proposed for quasi maximum likelihood blind source separation and blind deconvolution of one dimensional signals is generalized for blind deconvolution of images smooth approximation of the absolute value is used as the nonlinear term for sparse sources in addition we propose a method of sparsification which allows blind deconvolution of arbitrary sources and show how to find optimal sparsifying transformations by supervised learning

waveform preserving blind estimation of multiple independent sources the problem of blind estimation of source signals is to estimate the source signals without knowing the characteristics of the transmission channel it is shown that the minimum variance unbiased estimates can be obtained if and only if the transmission channel can be identified blindly it is shown that the channel can be blindly identified if and only if there is not more than one gaussian source this condition suggests that waveform preserving blind estimation can be achieved over a wide range of signal processing applications including those cases in which the source signals have identical nongaussian distributions the constructive proof of the necessary and sufficient condition serves as a foundation for the development of waveform preserving blind signal estimation algorithms examples are presented to demonstrate the applications of the theoretical results

independent component analysis by entropy bound minimization a novel differential entropy estimator is introduced where the maximum entropy bound is used to approximate the entropy given the observations and is computed using a numerical procedure thus resulting in accurate estimates for the entropy we show that such an estimator exists for a wide class of measuring functions and provide a number of design examples to demonstrate its flexible nature we then derive a novel independent component analysis ica algorithm that uses the entropy estimate thus obtained ica by entropy bound minimization ica ebm the algorithm adopts a line search procedure and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance we demonstrate the superior performance of ica ebm and its ability to match sources that come from a wide range of distributions using simulated and real world data

estimating functions of independent component analysis for temporally correlated signals this article studies a general theory of estimating functions of independent component analysis when the independent source signals are temporarily correlated estimating functions are used for deriving both batch and on line learning algorithms and they are applicable to blind cases where spatial and temporal probability structures of the sources are unknown most algorithms proposed so far can be analyzed in the framework of estimating functions an admissible class of estimating functions is derived and related efficient on line learning algorithms are introduced we analyze dynamical stability and statistical efficiency of these algorithms different from the independently and identically distributed case the algorithms work even when only the second order moments are used the method of simultaneous diagonalization of cross covariance matrices is also studied from the point of view of estimating functions

performance analysis of the fastica algorithm and crame spl acute r rao bounds for linear independent component analysis the fastica or fixed point algorithm is one of the most successful algorithms for linear independent component analysis ica in terms of accuracy and computational complexity two versions of the algorithm are available in literature and software a one unit deflation algorithm and a symmetric algorithm the main result of this paper are analytic closed form expressions that characterize the separating ability of both versions of the algorithm in a local sense assuming a good initialization of the algorithms and long data records based on the analysis it is possible to combine the advantages of the symmetric and one unit version algorithms and predict their performance to validate the analysis a simple check of saddle points of the cost function is proposed that allows to find a global minimum of the cost function in almost 100 simulation runs second the crame spl acute r rao lower bound for linear ica is derived as an algorithm independent limit of the achievable separation quality the fastica algorithm is shown to approach this limit in certain scenarios extensive computer simulations supporting the theoretical findings are included

globally convergent deflationary instantaneous blind source separation algorithm for digital communication signals recently an instantaneous blind source separation bss approach that exploits the bounded magnitude structure of digital communications signals has been introduced in this paper we introduce a deflationary adaptive algorithm based on this criterion and provide its convergence analysis we show that the resulting algorithm is convergent to one of the globally optimal points that correspond to perfect separation the simulation examples related to the separation of digital communication signals are provided to illustrate the convergence and the performance of the algorithm

blind source separation of signals with known alphabets using spl epsi approximation algorithms we show that blind separation of signals in given alphabets can be formulated into a quadratic optimization problem with integer constraints then efficient spl epsi approximation algorithms are applied to directly estimate the transmitted signals the proposed approach does not require any high order statistics moreover the algorithms converge to an spl epsi neighborhood of the global optimum with polynomial computational complexity simulations show that the algorithm achieves satisfactory performance using a short length of data

blind source separation by entropy rate minimization by assuming latent sources are statistically independent independent component analysis separates underlying sources from a given linear mixture since in many applications latent sources are both non gaussian and have sample dependence it is desirable to exploit both properties jointly in this paper we use mutual information rate to construct a general framework for analysis and derivation of algorithms that take both properties into account we discuss two types of source models for entropy rate estimation a markovian and an invertible filter model and give the general independent component analysis cost function update rule and performance analysis based on these we also introduce four algorithms based on these two models and show that their performance can approach the cramer rao lower bound in addition we demonstrate that the algorithms with flexible models exhibit very desirable performance for natural data

efficient independent component analysis independent component analysis ica has been widely used for blind source separation in many fields such as brain imaging analysis signal processing and telecommunication many statistical techniques based on m estimates have been proposed for estimating the mixing matrix recently several nonparametric methods have been developed but in depth analysis of asymptotic efficiency has not been available we analyze ica using semiparametric theories and propose a straightforward estimate based on the efficient score function by using b spline approximations the estimate is asymptotically efficient under moderate conditions and exhibits better performance than standard ica methods in a variety of simulations

source separation in post nonlinear mixtures we address the problem of separation of mutually independent sources in nonlinear mixtures first we propose theoretical results and prove that in the general case it is not possible to separate the sources without nonlinear distortion therefore we focus our work on specific nonlinear mixtures known as post nonlinear mixtures these mixtures constituted by a linear instantaneous mixture linear memoryless channel followed by an unknown and invertible memoryless nonlinear distortion are realistic models in many situations and emphasize interesting properties i e in such nonlinear mixtures sources can be estimated with the same indeterminacies as in instantaneous linear mixtures the separation structure of nonlinear mixtures is a two stage system namely a nonlinear stage followed by a linear stage the parameters of which are updated to minimize an output independence criterion expressed as a mutual information criterion the minimization of this criterion requires knowledge or estimation of source densities or of their log derivatives a first algorithm based on a gram charlier expansion of densities is proposed unfortunately it fails for hard nonlinear mixtures a second algorithm based on an adaptive estimation of the log derivative of densities leads to very good performance even with hard nonlinearities experiments are proposed to illustrate these results

fast and robust fixed point algorithms for independent component analysis independent component analysis ica is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible we use a combination of two different approaches for linear ica comon s information theoretic approach and the projection pursuit approach using maximum entropy approximations of differential entropy we introduce a family of new contrast functions for ica these contrast functions enable both the estimation of the whole decomposition by minimizing mutual information and estimation of individual independent components as projection pursuit directions the statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model and it is shown how to choose contrast functions that are robust and or of minimum variance finally we introduce simple fixed point algorithms for practical optimization of the contrast functions

nonsymmetrical contrasts for sources separation in this paper the problem of the blind separation of independent sources is considered our approach relies on high order inverse criteria after generalizing the definition of classical contrast functions we exhibit a wide class of generally nonsymmetrical functions that will be called generalized contrasts and whose maximization is proved to be a sufficient condition for source separation we also establish a connection with cumulant matching showing an equivalence between the two approaches then in the case of two sources a statistical study of the estimated parameter based on one of these new contrasts is presented finally computer simulations illustrate the results and demonstrate all the interest we can find in considering a nonsymmetrical contrast

source extraction by maximizing the variance in the conditional distribution tails this paper presents a method for signal extraction based on conditional second order moments of the output of the extraction filter the estimator of the filter is derived from an approximate maximum likelihood criterion conditioned on a presence indicator of the source of interest the conditional moment is shown to be a contrast function under the conditions that 1 all cross moments of the same order between the source signal of interest and the other source signals are null and 2 that the source of interest has the largest conditional moment among all sources for the two source two observation case this allows us to derive the theoretical recovery bounds of the contrast when the conditional cross moment does not vanish a comparison with empirical results confirms these bounds simulations show that the estimator is quite robust to additive gaussian distributed noise also through simulations we show that the error level induced by a rough approximation of the presence indicator shows a strong similarity with that of additive noise the robustness with respect both to noise and to inaccuracies in the prior information about the source presence guarantees a wide applicability of the proposed method

estimation of a structural vector autoregression model using non gaussianity analysis of causal effects between continuous valued variables typically uses either autoregressive models or structural equation models with instantaneous effects estimation of gaussian linear structural equation models poses serious identifiability problems which is why it was recently proposed to use non gaussian models here we show how to combine the non gaussian instantaneous model with autoregressive models this is effectively what is called a structural vector autoregression svar model and thus our work contributes to the long standing problem of how to estimate svar s we show that such a non gaussian model is identifiable without prior knowledge of network structure we propose computationally efficient methods for estimating the model as well as methods to assess the significance of the causal influences the model is successfully applied on financial and brain imaging data

criteria for multichannel signal separation we consider the problem in which we want to separate two or more signals that are coupled to each other through an unknown multiple input multiple output linear system channel we prove that the signals can be decoupled or separated using only the condition that they are statistically independent and find even weaker sufficient conditions involving their cross polyspectra by imposing these conditions on the reconstructed signals we obtain a class of criteria for signal separation these criteria are universal in the sense that they do not require any prior knowledge or information concerning the nature of the source signals they may be communication signals or speech signals or any other 1 d or multidimensional signals e g images computationally efficient algorithms for implementing the proposed criteria that only involve the iterative solution to a linear least squares problem are presented

an expectation maximization method for spatio temporal blind source separation using an ar mog source model in this paper we develop a maximum likelihood ml spatio temporal blind source separation bss algorithm where the temporal dependencies are explained by assuming that each source is an autoregressive ar process and the distribution of the associated independent identically distributed i i d innovations process is described using a mixture of gaussians unlike most ml methods the proposed algorithm takes into account both spatial and temporal information optimization is performed using the expectation maximization em method the source model is adapted to maximize the likelihood and the update equations have a simple analytical form the proposed method which we refer to as autoregressive mixture of gaussians ar mog outperforms nine other methods for artificial mixtures of real audio we also show results for using ar mog to extract the fetal cardiac signal from real magnetocardiographic mcg data

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

a linear non gaussian acyclic model for causal discovery in recent years several methods have been proposed for the discovery of causal structure from non experimental data such methods make various assumptions on the data generating process to facilitate its identification from purely observational data continuing this line of research we show how to discover the complete causal structure of continuous valued data under the assumptions that a the data generating process is linear b there are no unobserved confounders and c disturbance variables have non gaussian distributions of non zero variances the solution relies on the use of the statistical method known as independent component analysis and does not require any pre specified time ordering of the variables we provide a complete matlab package for performing this lingam analysis short for linear non gaussian acyclic model and demonstrate the effectiveness of the method using artificially generated data and real world data

a blind source separation technique using second order statistics separation of sources consists of recovering a set of signals of which only instantaneous linear mixtures are observed in many situations no a priori information on the mixing matrix is available the linear mixture should be blindly processed this typically occurs in narrowband array processing applications when the array manifold is unknown or distorted this paper introduces a new source separation technique exploiting the time coherence of the source signals in contrast with other previously reported techniques the proposed approach relies only on stationary second order statistics that are based on a joint diagonalization of a set of covariance matrices asymptotic performance analysis of this method is carried out some numerical simulations are provided to illustrate the effectiveness of the proposed method

hidden markov models for wavelet based blind source separation in this paper we consider the problem of blind source separation in the wavelet domain we propose a bayesian estimation framework for the problem where different models of the wavelet coefficients are considered the independent gaussian mixture model the hidden markov tree model and the contextual hidden markov field model for each of the three models we give expressions of the posterior laws and propose appropriate markov chain monte carlo algorithms in order to perform unsupervised joint blind separation of the sources and estimation of the mixing matrix and hyper parameters of the problem indeed in order to achieve an efficient joint separation and denoising procedures in the case of high noise level in the data a slight modification of the exposed models is presented the bernoulli gaussian mixture model which is equivalent to a hard thresholding rule in denoising problems a number of simulations are presented in order to highlight the performances of the aforementioned approach 1 in both high and low signal to noise ratios and 2 comparing the results with respect to the choice of the wavelet basis decomposition

a simple geometric blind source separation method for bounded magnitude sources a novel blind source separation approach and the corresponding adaptive algorithm is presented it is assumed that the observation mixture is obtained through an unknown memoryless linear mapping of independent and bounded magnitude sources we further assume an initial adaptive prewhitening of the original observation vector which transforms it into a white vector with the same dimension as the original source vector our approach is centered around the basic geometric fact that under a certain boundedness assumption the unitary mapping which transforms the whitening output vector into an independent vector has the minimum value of maximum real component magnitude output over the ensemble of all output components therefore the related criterion is the minimization of the infinity norm of the real component of the unitary separator s output over all possible output combinations for the minimization of the corresponding nondifferentiable cost function we propose the use of subgradient optimization methods to obtain a low complexity iterative adaptive solution the resulting algorithm is fairly intuitive and simple and provides a low complexity solution especially to a class of multiuser digital communications problems we provide examples at the end of this paper to illustrate the performance of our algorithm

markovian source separation a maximum likelihood ml approach is used to separate the instantaneous mixtures of temporally correlated independent sources with neither preliminary transformation nor a priori assumption about the probability distribution of the sources a markov model is used to represent the joint probability density of successive samples of each source the joint probability density functions are estimated from the observations using a kernel method for the special case of autoregressive models the theoretical performance of the algorithm is computed and compared with the performance of second order algorithms and i i d based separation algorithms

high order contrasts for independent component analysis this article considers high order measures of independence for the independent component analysis problem and discusses the class of jacobi algorithms for their optimization several implementations are discussed we compare the proposed approaches with gradient based techniques from the algorithmic point of view and also on a set of biomedical data

blind source separation by sparse decomposition in a signal dictionary the blind source separation problem is to extract the underlying source signals from a set of linear mixtures where the mixing matrix is unknown this situation is common in acoustics radio medical signal and image processing hyperspectral imaging and other areas we suggest a two stage separation process a priori selection of a possibly overcomplete signal dictionary for instance a wavelet frame or a learned dictionary in which the sources are assumed to be sparsely representable followed by unmixing the sources by exploiting the their sparse representability we consider the general case of more sources than mixtures but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques

identification and estimation of non gaussian structural vector autoregressions abstract conventional structural vector autoregressive svar models with gaussian errors are not identified and additional identifying restrictions are needed in applied work we show that the gaussian case is an exception in that a svar model whose error vector consists of independent non gaussian components is without any additional restrictions identified and leads to essentially unique impulse responses building upon this result we introduce an identification scheme under which the maximum likelihood estimator of the parameters of the non gaussian svar model is consistent and asymptotically normally distributed as a consequence additional economic identifying restrictions can be tested in an empirical application we find a negative impact of a contractionary monetary policy shock on financial markets and clearly reject the commonly employed recursive identifying restrictions

a coordinate descent algorithm for complex joint diagonalization under hermitian and transpose congruences abstract this paper deals with the problem of joint complexmatrix diagonalization by considering both the hermitian andtranspose congruences we address the general case where thesearched diagonalizing matrix is apriori nonunitary based onthe minimization of a quadratic criterion we propose a ex ible algorithm in the sense that it allows to directly consider arectangular diagonalizing matrix and to take into considerationboth the hermitian and transpose congruences within the sameframework the proposed algorithm is a coordinate descentalgorithm that is based on an approximate criterion leading to theanalytical derivation of the minima arguments computer simu lations are drawn to illustrate the usefulness and performancesof the algorithm and a comparison to state of the art algorithmsis presented finally an application to independent componentanalysis based on fourth order statistics is also presented index terms blind source separation complex symmetric ma trices hermitian matrices independent component analysis jointdiagonalization

blind decomposition of transmission light microscopic hyperspectral cube using sparse representation in this paper we address the problem of fully automated decomposition of hyperspectral images for transmission light microscopy the hyperspectral images are decomposed into spectrally homogeneous compounds the resulting compounds are described by their spectral characteristics and optical density we present the multiplicative physical model of image formation in transmission light microscopy justify reduction of a hyperspectral image decomposition problem to a blind source separation problem and provide method for hyperspectral restoration of separated compounds in our approach dimensionality reduction using principal component analysis pca is followed by a blind source separation bss algorithm the bss method is based on sparsifying transformation of observed images and relative newton optimization procedure the presented method was verified on hyperspectral images of biological tissues the method was compared to the existing approach based on nonnegative matrix factorization experiments showed that the presented method is faster and better separates the biological compounds from imaging artifacts the results obtained in this work may be used for improving automatic microscope hardware calibration and computer aided diagnostics

faster independent component analysis by preconditioning with hessian approximations independent component analysis ica is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences in its classic form ica relies on modeling the data as linear mixtures of nongaussian independent sources the maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data this problem has been addressed by resorting to quasi newton methods which rely on sparse approximations of the hessian of the log likelihood however those approximations are not accurate when the ica model does not hold exactly as is often the case for real datasets we propose a new algorithm dubbed picard which makes use of sparse approximate hessians only as a preconditioner to the l bfgs algorithm refining the hessian approximation from a memory of the past iterates extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique especially on real data

statistical inference for independent component analysis application to structural var models abstract the well known problem of non identifiability of structural var models disappears if the structural shocks are independent and if at most one of them is gaussian in that case the relevant estimation technique is the independent component analysis ica since the introduction of ica by comon 1994 various semi parametric estimation methods have been proposed for orthogonalizing the error terms these methods include pseudo maximum likelihood pml approaches and recursive pml the aim of our paper is to derive the asymptotic properties of the pml approaches in particular to study their consistency we conduct monte carlo studies exploring the relative performances of these methods finally an application based on real data shows that structural var models can be estimated without additional identification restrictions in the non gaussian case and that the usual restrictions can be tested

a note on redundancy averaging it is well known that the performance of direction of arrival estimation and beamforming algorithms degrades in the presence of correlated signals since techniques like spatial and weighted smoothing which were developed for decorrelating the signals suffer from reduced effective aperture several authors have recently proposed redundancy averaging as an alternative spatial averaging method considering a two source signal model and the asymptotic case the authors show analytically that this technique results in an eigenstructure which is inconsistent with that of the underlying signal model in particular the resulting signal subspace fills up the whole m dimensional space where m is the array size and the resulting covariance matrix is not guaranteed to be nonnegative definite it is shown that complete decorrelation is not possible with this method even if the array size is infinitely large

gradient algorithms for complex non gaussian independent component vector extraction question of convergence we revise the problem of extracting one independent component from an instantaneous linear mixture of signals the mixing matrix is parameterized by two vectors one column of the mixing matrix and one row of the demixing matrix the separation is based on the non gaussianity of the source of interest while the remaining background signals are assumed to be gaussian three gradient based estimation algorithms are derived using the maximum likelihood principle and are compared with the natural gradient algorithm for independent component analysis and with one unit fastica based on negentropy maximization the ideas and algorithms are also generalized to the extraction of a vector component when the extraction proceeds jointly from a set of instantaneous mixtures throughout this paper we address the problem concerning the size of the region of convergence for which the algorithms guarantee the extraction of the desired source we show that the size is influenced by the signal to interference ratio on the input channels simulations comparing several algorithms confirm this observation they show a different size of the region of convergence under a scenario in which the source of interest is dominant or weak here our proposed modifications of the gradient methods taking into account the dominance weakness of the source show improved global convergence

superefficiency in blind source separation blind source separation is the problem of extracting independent signals from their mixtures without knowing the mixing coefficients nor the probability distributions of source signals and may be applied to eeg and meg imaging of the brain it is already known that certain algorithms work well for the extraction of independent components the present paper is concerned with superefficiency of these based on the statistical and dynamical analysis in a statistical estimation using t examples the covariance of any two extracted independent signals converges to 0 of the order of 1 t on line dynamics shows that the covariance is of the order of spl eta when the learning rate spl eta is fixed to a small constant in contrast with the above general properties a surprising superefficiency holds in blind source separation under certain conditions where superefficiency implies that covariance decreases in the order of 1 t sup 2 or of spl eta sup 2 the paper uses the natural gradient learning algorithm and method of estimating functions to obtain superefficient procedures for both batch estimation and on line learning a standardized estimating function is introduced to this end superefficiency does not imply that the error variances of the extracted signals decrease in the order of 1 t sup 2 or spl eta sup 2 but implies that their covariances and independencies do