performance analysis of the gaussian quasi maximum likelihood approach for independent vector analysis maximum likelihood ml estimation requires precise knowledge of the underlying statistical model in quasi ml qml a presumed model is used as a substitute to the unknown true model in the context of independent vector analysis iva we consider the gaussian qml estimate qmle of the demixing matrices set and present an approximate analysis of its asymptotic separation performance in gaussian qml the sources are presumed to be gaussian with covariance matrices specified by some educated guess the resulting quasi likelihood equations of the demixing matrices take a special form recently termed an extended sequentially drilled joint congruence sedjoco transformation which is reminiscent of though essentially different from classical joint diagonalization we show that asymptotically this qmle i e the solution of the resulting extended sedjoco transformation attains perfect separation under some mild conditions regardless of the sources true distributions and or covariance matrices in addition based on the small errors assumption we present a first order perturbation analysis of the extended sedjoco solution using the resulting closed form expressions for the errors in the solution matrices we provide closed form expressions for the resulting interference to source ratios isrs for iva moreover we prove that asymptotically the isrs depend only on the sources covariances and not on their specific distributions as an immediate consequence of this result we provide an asymptotically attainable lower bound on the resulting isrs we also present empirical results corroborating our analytical derivations of three simulation experiments concerning two possible model errors inaccurate covariance matrices and sources distribution mismodeling

non self adjoint hamiltonians defined by riesz bases we discuss some features of non self adjoint hamiltonians with real discrete simple spectrum under the assumption that the eigenvectors form a riesz basis of hilbert space among other things we give conditions under which these hamiltonians can be factorized in terms of generalized lowering and raising operators

compact cramer rao bound expression for independent component analysis despite of the increased interest in independent component analysis ica during the past two decades a simple closed form expression of the cramer rao bound crb for the demixing matrix estimation has not been established in the open literature in the present paper we fill this gap by deriving a simple closed form expression for the crb of the demixing matrix directly from its definition a simulation study comparing ica estimators with the crb is given

asymmetric cubature formulae with few points in high dimension for symmetric measures let mu be a positive measure on mathbb r d invariant under the group of reflections and permutations and let m be a natural number we describe a method to construct cubature formulae of degree m with respect to mu with n positive weights and n points in the support of mu and such that n grows at most like dm with the dimension d we apply this method to classical measures to explicitly construct cubature formulae of degree 5 with the number of points growing at most like d3

a generalized polynomial chaos based ensemble kalman filter with high accuracy as one of the most adopted sequential data assimilation methods in many areas especially those involving complex nonlinear dynamics the ensemble kalman filter enkf has been under extensive investigation regarding its properties and efficiency compared to other variants of the kalman filter kf enkf is straightforward to implement as it employs random ensembles to represent solution states this however introduces sampling errors that affect the accuracy of enkf in a negative manner though sampling errors can be easily reduced by using a large number of samples in practice this is undesirable as each ensemble member is a solution of the system of state equations and can be time consuming to compute for large scale problems in this paper we present an efficient enkf implementation via generalized polynomial chaos gpc expansion the key ingredients of the proposed approach involve 1 solving the system of stochastic state equations via the gpc methodology to gain efficiency and 2 sampling the gpc approximation of the stochastic solution with an arbitrarily large number of samples at virtually no additional computational cost to drastically reduce the sampling errors the resulting algorithm thus achieves a high accuracy at reduced computational cost compared to the classical implementations of enkf numerical examples are provided to verify the convergence property and accuracy improvement of the new algorithm we also prove that for linear systems with gaussian noise the first order gpc kalman filter method is equivalent to the exact kalman filter

an iterative inversion approach to blind source separation we present an iterative inversion ii approach to blind source separation bss it consists of a quasi newton method for the resolution of an estimating equation obtained from the implicit inversion of a robust estimate of the mixing system the resulting learning rule includes several existing algorithms for bss as particular cases giving them a novel and unified interpretation it also provides a justification of the cardoso and laheld 1996 step size normalization the ii method is first presented for instantaneous mixtures and then extended to the problem of blind separation of convolutive mixtures finally we derive the necessary and sufficient asymptotic stability conditions for both the instantaneous and convolutive methods to converge

joint independent subspace analysis using second order statistics this paper deals with a novel generalization of classical blind source separation bss in two directions first relaxing the constraint that the latent sources must be statistically independent this generalization is well known and sometimes termed independent subspace analysis isa second jointly analyzing several isa problems where the link is due to statistical dependence among corresponding sources in different mixtures when the data are one dimensional i e multiple classical bss problems this model known as independent vector analysis iva has already been studied in this paper we combine iva with isa and term this new model joint independent subspace analysis jisa we provide full performance analysis of jisa including closed form expressions for minimal mean square error mse fisher information and cramer rao lower bound in the separation of gaussian data the derived mse applies also for non gaussian data when only second order statistics are used we generalize previously known results on iva including its ability to uniquely resolve instantaneous mixtures of real gaussian stationary data and having the same arbitrary permutation at all mixtures numerical experiments validate our theoretical results and show the gain with respect to two competing approaches that either use a finer block partition or a different norm

blind separation of instantaneous mixture of sources via an independent component analysis in this paper we introduce a procedure for separating a multivariate distribution into nearly independent components based on minimizing a criterion defined in terms of the kullback leibner distance by replacing the unknown density with a kernel estimate we derive useful forms of this criterion when only a sample from that distribution is available we also compute the gradient and hessian of our criteria for use in an iterative minimization setting this gradient to zero yields a set of separating functions similar to the ones considered in the source separation problem except that here these functions are adapted to the observed data finally some simulations are given illustrating the good performance of the method

quasi maximum likelihood mimo blind deconvolution super and sub gaussianity versus consistency in this correspondence we consider the problem of multi input multi output mimo quasi maximum likelihood qml blind deconvolution we examine two classes of estimators which are commonly believed to be suitable for super and sub gaussian sources we state the consistency conditions and demonstrate a source distribution for which the studied estimators are unsuitable in the sense that they are inconsistent

adaptive approximation of higher order posterior statistics filtering is an approach for incorporating observed data into time evolving systems instead of a family of dirac delta masses that is widely used in monte carlo methods we here use the wiener chaos expansion for the parametrization of the conditioned probability distribution to solve the nonlinear filtering problem the wiener chaos expansion is not the best method for uncertainty propagation without observations nevertheless the projection of the system variables in a fixed polynomial basis spanning the probability space might be a competitive representation in the presence of relatively frequent observations because the wiener chaos approach not only leads to an accurate and efficient prediction for short time uncertainty quantification but it also allows to apply several data assimilation methods that can be used to yield a better approximate filtering solution the aim of the present paper is to investigate this hypothesis we answer in the affirmative for the stochastic lorenz 63 system based on numerical simulations in which the uncertainty quantification method and the data assimilation method are adaptively selected by whether the dynamics is driven by brownian motion and the near gaussianity of the measure to be updated respectively

robust independent component analysis by iterative maximization of the kurtosis contrast with algebraic optimal step size independent component analysis ica aims at decomposing an observed random vector into statistically independent variables deflation based implementations such as the popular one unit fastica algorithm and its variants extract the independent components one after another a novel method for deflationary ica referred to as robustica is put forward in this paper this simple technique consists of performing exact line search optimization of the kurtosis contrast function the step size leading to the global maximum of the contrast along the search direction is found among the roots of a fourth degree polynomial this polynomial rooting can be performed algebraically and thus at low cost at each iteration among other practical benefits robustica can avoid prewhitening and deals with real and complex valued mixtures of possibly noncircular sources alike the absence of prewhitening improves asymptotic performance the algorithm is robust to local extrema and shows a very high convergence speed in terms of the computational cost required to reach a given source extraction quality particularly for short data records these features are demonstrated by a comparative numerical analysis on synthetic data robustica s capabilities in processing real world data involving noncircular complex strongly super gaussian sources are illustrated by the biomedical problem of atrial activity aa extraction in atrial fibrillation af electrocardiograms ecgs where it outperforms an alternative ica based technique

blind separation of parametric nonlinear mixtures of possibly autocorrelated and non stationary sources in this paper we present a new method formulated in a maximum likelihood framework for blindly separating nonlinear mixtures of statistically independent signals our method exploits on the one hand the knowledge of the parametric model of the mixing transformation with unknown parameter values and on the other hand the possible structure of source signals i e their autocorrelation and or nonstationarity one of the main advantages of the proposed method is that it can be implemented even if the analytical expression of the inverse model is unknown the method is first addressed in a general configuration then detailed for two special cases i e a simple bijective toy model and a linear quadratic model the study of the toy model is interesting because of its simplicity and its global bijectivity which allows us to focus our efforts on parameter estimation the linear quadratic model is chosen due to its capacity to describe real world mixing phenomena simulation results using the toy model and using a subclass of the linear quadratic model i e the bilinear model show that taking into account the nonlinearity of the mixing transformations and the structure of signals considerably improves separation performance

non self adjoint hamiltonians defined by generalized riesz bases bagarello inoue and trapani j math phys 55 033501 2014 investigated some operators defined by the riesz bases these operators connect with quasi hermitian quantum mechanics and its relatives in this paper we introduce a notion of generalized riesz bases which is a generalization of riesz bases and investigate some operators defined by the generalized riesz bases by changing the frameworks of the operators defined in the work of bagarello inoue and trapani

more mathematics for pseudo bosons we propose an alternative definition for pseudo bosons this simplifies the mathematical structure minimizing the required assumptions some physical examples are discussed as well as some mathematical results related to the biorthogonal sets arising out of our framework we also briefly extend the results to the so called nonlinear pseudo bosons

exact maximum likelihood parameter estimation of superimposed exponential signals in noise a unified framework for the exact maximum likelihood estimation of the parameters of superimposed exponential signals in noise encompassing both the time series and the array problems is presented an exact expression for the ml criterion is derived in terms of the linear prediction polynomial of the signal and an iterative algorithm for the maximization of this criterion is presented the algorithm is equally applicable in the case of signal coherence in the array problem simulation shows the estimator to be capable of providing more accurate frequency estimates than currently existing techniques the algorithm is similar to those independently derived by kumaresan et al in addition to its practical value the present formulation is used to interpret previous methods such as prony s pisarenko s and modifications thereof

high order collocation methods for differential equations with random inputs recently there has been a growing interest in designing efficient methods for the solution of ordinary partial differential equations with random inputs to this end stochastic galerkin methods appear to be superior to other nonsampling methods and in many cases to several sampling methods however when the governing equations take complicated forms numerical implementations of stochastic galerkin methods can become nontrivial and care is needed to design robust and efficient solvers for the resulting equations on the other hand the traditional sampling methods e g monte carlo methods are straightforward to implement but they do not offer convergence as fast as stochastic galerkin methods in this paper a high order stochastic collocation approach is proposed similar to stochastic galerkin methods the collocation methods take advantage of an assumption of smoothness of the solution in random space to achieve fast convergence however the numerical implementation of stochastic collocation is trivial as it requires only repetitive runs of an existing deterministic solver similar to monte carlo methods the computational cost of the collocation methods depends on the choice of the collocation points and we present several feasible constructions one particular choice based on sparse grids depends weakly on the dimensionality of the random space and is more suitable for highly accurate computations of practical applications with large dimensional random inputs numerical examples are presented to demonstrate the accuracy and efficiency of the stochastic collocation methods

deconvolution of multiple time series the problem of estimating simultaneously two convolved vector time series corrupted by additive noise is considered by regarding one of the series as being stochastic and the other as fixed it is shown that the fixed component can be estimated by maximizing a frequency domain approximation to the likelihood the stochastic series is estimated by using an approximation to the conditional mean evaluated at the current maximum likelihood estimators an example involving a multiple deconvolution of seismic source and receiver functions is given

kernel methods for measuring independence we introduce two new functionals the constrained covariance and the kernel mutual information to measure the degree of independence of random variables these quantities are both based on the covariance between functions of the random variables in reproducing kernel hilbert spaces rkhss we prove that when the rkhss are universal both functionals are zero if and only if the random variables are pairwise independent we also show that the kernel mutual information is an upper bound near independence on the parzen window estimate of the mutual information analogous results apply for two correlation based dependence functionals introduced earlier we show the kernel canonical correlation and the kernel generalised variance to be independence measures for universal kernels and prove the latter to be an upper bound on the mutual information near independence the performance of the kernel dependence functionals in measuring independence is verified in the context of independent component analysis

relative optimization for blind deconvolution we propose a relative optimization framework for quasi maximum likelihood qml blind deconvolution and the relative newton method as its particular instance special hessian structure allows fast newton system construction and solution resulting in a fast convergent algorithm with iteration complexity comparable to that of gradient methods we also propose the use of rational infinite impulse response iir restoration kernels which constitute a richer family of filters than the traditionally used finite impulse response fir kernels we discuss different choices of nonlinear functions that are suitable for deconvolution of super and sub gaussian sources and formulate the conditions under which the qml estimation is stable simulation results demonstrate the efficiency of the proposed methods

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

algorithms for complex ml ica and their stability analysis using wirtinger calculus we derive a class of algorithms for independent component analysis ica based on maximum likelihood ml estimation and perform stability analysis of natural gradient ml ica with and without the constraint for unitary demixing matrix in the process we demonstrate how wirtinger calculus facilitates derivations and most importantly performing second order analysis in the complex domain and eliminates the need for making simplifying assumptions we derive natural gradient complex ml ica update rule and its variant with a unitary constraint as well as a newton algorithm for better convergence behavior the conditions for local stability are derived and studied using a generalized gaussian density ggd source model when the sources are circular and non gaussian we show analytically that both the ml and ml unitary ica update rules converge to the inverse of mixing matrix subject to a phase shift when the sources are noncircular and non gaussian we show that the nonunitary ml ica update rule is more stable than the ml unitary ica update rule when the sources are noncircular gaussians both update rules are stable only when the sources have distinct noncircularity indices simulation results are given to support these results

general theory of regular biorthogonal pairs and its physical operators in this paper we introduce a general theory of regular biorthogonal sequences and its physical operators biorthogonal sequences n and n in a hilbert space h are said to be regular if span n and span n are dense in h the first purpose is to show that there exists a non singular positive self adjoint operator tf in h defined by an orthonormal basis onb f fn in h such that n tffn and n tf 1fn n 0 1 and such an onb f is unique the second purpose is to define and study the lowering operators af and bf the raising operators bf and af and the number operators nf and nf determined by the non singular positive self adjoint operator tf these operators connect with quasi hermitian quantum mechanics and its relatives this paper clarifies and simplifies the mathematical structure of this framework and minimizes the required assumptions

frequency domain blind deconvolution based on mutual information rate in this paper a new blind single input single output siso deconvolution method based on the minimization of the mutual information rate of the deconvolved output is proposed the method works in the frequency domain and requires estimation of the signal probability density function thus the algorithm uses higher order statistics except for gaussian source and allows non minimum phase filter estimation in practice the criterion contains a regularization term for limiting noise amplification as in wiener filtering the score function estimation which represents a key point of the algorithm is detailed and the most robust estimate is selected finally experiments point to the relevance of the proposed algorithm 1 any filter minimum phase or not can be estimated and 2 on actual data underwater explosions seismovolcanic phenomena this deconvolution algorithm provides good results with a better tradeoff between deconvolution quality and noise amplification than existing methods

blind deconvolution of images using optimal sparse representations the relative newton algorithm previously proposed for quasi maximum likelihood blind source separation and blind deconvolution of one dimensional signals is generalized for blind deconvolution of images smooth approximation of the absolute value is used as the nonlinear term for sparse sources in addition we propose a method of sparsification which allows blind deconvolution of arbitrary sources and show how to find optimal sparsifying transformations by supervised learning

a direct solution for blind separation of sources proposes a direct estimation of the mixing matrix in the source separation problem as a new result we prove that for non gaussian sources the mixing matrix can be obtained by solving a fourth degree polynomial equation that derives from nonlinear equations only involving fourth order cumulants

a sequentially drilled joint congruence sedjoco transformation with applications in blind source separation and multiuser mimo systems we consider a particular form of the classical approximate joint diagonalization ajd problem which we call a sequentially drilled joint congruence sedjoco transformation the problem consists of a set of symmetric real valued or hermitian symmetric complex valued target matrices the number of matrices in the set equals their dimension and the joint diagonality criterion requires that in each transformed diagonalized target matrix all off diagonal elements on one specific row and column corresponding to the matrix index in the set be exactly zeros yet does not care about the other diagonal or off diagonal elements the motivation for this form arises in at least two different contexts maximum likelihood blind or semiblind source separation and coordinated beamforming for multiple input multiple output mimo broadcast channels we prove that sedjoco always has a solution when the target matrices are positive definite we also propose two possible iterative solution algorithms based on defining and optimizing two different criteria functions using newton s method for the first function and successive jacobi like transformations for the second the algorithms convergence behavior and the attainable performance in the two contexts above are demonstrated in simulation experiments

waveform preserving blind estimation of multiple independent sources the problem of blind estimation of source signals is to estimate the source signals without knowing the characteristics of the transmission channel it is shown that the minimum variance unbiased estimates can be obtained if and only if the transmission channel can be identified blindly it is shown that the channel can be blindly identified if and only if there is not more than one gaussian source this condition suggests that waveform preserving blind estimation can be achieved over a wide range of signal processing applications including those cases in which the source signals have identical nongaussian distributions the constructive proof of the necessary and sufficient condition serves as a foundation for the development of waveform preserving blind signal estimation algorithms examples are presented to demonstrate the applications of the theoretical results

independent component analysis by entropy bound minimization a novel differential entropy estimator is introduced where the maximum entropy bound is used to approximate the entropy given the observations and is computed using a numerical procedure thus resulting in accurate estimates for the entropy we show that such an estimator exists for a wide class of measuring functions and provide a number of design examples to demonstrate its flexible nature we then derive a novel independent component analysis ica algorithm that uses the entropy estimate thus obtained ica by entropy bound minimization ica ebm the algorithm adopts a line search procedure and initially uses updates that constrain the demixing matrix to be orthogonal for robust performance we demonstrate the superior performance of ica ebm and its ability to match sources that come from a wide range of distributions using simulated and real world data

estimating functions of independent component analysis for temporally correlated signals this article studies a general theory of estimating functions of independent component analysis when the independent source signals are temporarily correlated estimating functions are used for deriving both batch and on line learning algorithms and they are applicable to blind cases where spatial and temporal probability structures of the sources are unknown most algorithms proposed so far can be analyzed in the framework of estimating functions an admissible class of estimating functions is derived and related efficient on line learning algorithms are introduced we analyze dynamical stability and statistical efficiency of these algorithms different from the independently and identically distributed case the algorithms work even when only the second order moments are used the method of simultaneous diagonalization of cross covariance matrices is also studied from the point of view of estimating functions

performance analysis of the fastica algorithm and crame spl acute r rao bounds for linear independent component analysis the fastica or fixed point algorithm is one of the most successful algorithms for linear independent component analysis ica in terms of accuracy and computational complexity two versions of the algorithm are available in literature and software a one unit deflation algorithm and a symmetric algorithm the main result of this paper are analytic closed form expressions that characterize the separating ability of both versions of the algorithm in a local sense assuming a good initialization of the algorithms and long data records based on the analysis it is possible to combine the advantages of the symmetric and one unit version algorithms and predict their performance to validate the analysis a simple check of saddle points of the cost function is proposed that allows to find a global minimum of the cost function in almost 100 simulation runs second the crame spl acute r rao lower bound for linear ica is derived as an algorithm independent limit of the achievable separation quality the fastica algorithm is shown to approach this limit in certain scenarios extensive computer simulations supporting the theoretical findings are included

blind source separation of signals with known alphabets using spl epsi approximation algorithms we show that blind separation of signals in given alphabets can be formulated into a quadratic optimization problem with integer constraints then efficient spl epsi approximation algorithms are applied to directly estimate the transmitted signals the proposed approach does not require any high order statistics moreover the algorithms converge to an spl epsi neighborhood of the global optimum with polynomial computational complexity simulations show that the algorithm achieves satisfactory performance using a short length of data

non self adjoint hamiltonians defined by sesquilinear forms and their physical applications in this paper we introduce a theory of sesquilinear forms with respect to a non self adjoint hamiltonian and its physical operators in particular we discuss some features of non self adjoint operators and sesquilinear forms which are defined starting from biorthogonal sequences that are not riesz bases generalized riesz systems d quasi bases and d e quasi bases

fast and robust fixed point algorithms for independent component analysis independent component analysis ica is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible we use a combination of two different approaches for linear ica comon s information theoretic approach and the projection pursuit approach using maximum entropy approximations of differential entropy we introduce a family of new contrast functions for ica these contrast functions enable both the estimation of the whole decomposition by minimizing mutual information and estimation of individual independent components as projection pursuit directions the statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model and it is shown how to choose contrast functions that are robust and or of minimum variance finally we introduce simple fixed point algorithms for practical optimization of the contrast functions

nonsymmetrical contrasts for sources separation in this paper the problem of the blind separation of independent sources is considered our approach relies on high order inverse criteria after generalizing the definition of classical contrast functions we exhibit a wide class of generally nonsymmetrical functions that will be called generalized contrasts and whose maximization is proved to be a sufficient condition for source separation we also establish a connection with cumulant matching showing an equivalence between the two approaches then in the case of two sources a statistical study of the estimated parameter based on one of these new contrasts is presented finally computer simulations illustrate the results and demonstrate all the interest we can find in considering a nonsymmetrical contrast

source extraction by maximizing the variance in the conditional distribution tails this paper presents a method for signal extraction based on conditional second order moments of the output of the extraction filter the estimator of the filter is derived from an approximate maximum likelihood criterion conditioned on a presence indicator of the source of interest the conditional moment is shown to be a contrast function under the conditions that 1 all cross moments of the same order between the source signal of interest and the other source signals are null and 2 that the source of interest has the largest conditional moment among all sources for the two source two observation case this allows us to derive the theoretical recovery bounds of the contrast when the conditional cross moment does not vanish a comparison with empirical results confirms these bounds simulations show that the estimator is quite robust to additive gaussian distributed noise also through simulations we show that the error level induced by a rough approximation of the presence indicator shows a strong similarity with that of additive noise the robustness with respect both to noise and to inaccuracies in the prior information about the source presence guarantees a wide applicability of the proposed method

a survey of convergence results on particle filtering methods for practitioners optimal filtering problems are ubiquitous in signal processing and related fields except for a restricted class of models the optimal filter does not admit a closed form expression particle filtering methods are a set of flexible and powerful sequential monte carlo methods designed to solve the optimal filtering problem numerically the posterior distribution of the state is approximated by a large set of dirac delta masses samples particles that evolve randomly in time according to the dynamics of the model and the observations the particles are interacting thus classical limit theorems relying on statistically independent samples do not apply in this paper our aim is to present a survey of convergence results on this class of methods to make them accessible to practitioners

regular pseudo bosons versus bosons we discuss in which sense the so called regular pseudo bosons recently introduced by trifonov and analyzed in some detail by the author are related to ordinary bosons we repeat the same analysis for pseudo bosons and analyze the role played by certain intertwining operators which may be bounded or not

monomial cubature rules since stroud a compilation a bibliography of references to cubature rules which have appeared since the publication of stroud s book 1971 is presented the standard regions that are treated in this paper are the n cube the n simplex the n sphere and the entire space

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

criteria for multichannel signal separation we consider the problem in which we want to separate two or more signals that are coupled to each other through an unknown multiple input multiple output linear system channel we prove that the signals can be decoupled or separated using only the condition that they are statistically independent and find even weaker sufficient conditions involving their cross polyspectra by imposing these conditions on the reconstructed signals we obtain a class of criteria for signal separation these criteria are universal in the sense that they do not require any prior knowledge or information concerning the nature of the source signals they may be communication signals or speech signals or any other 1 d or multidimensional signals e g images computationally efficient algorithms for implementing the proposed criteria that only involve the iterative solution to a linear least squares problem are presented

a linear non gaussian acyclic model for causal discovery in recent years several methods have been proposed for the discovery of causal structure from non experimental data such methods make various assumptions on the data generating process to facilitate its identification from purely observational data continuing this line of research we show how to discover the complete causal structure of continuous valued data under the assumptions that a the data generating process is linear b there are no unobserved confounders and c disturbance variables have non gaussian distributions of non zero variances the solution relies on the use of the statistical method known as independent component analysis and does not require any pre specified time ordering of the variables we provide a complete matlab package for performing this lingam analysis short for linear non gaussian acyclic model and demonstrate the effectiveness of the method using artificially generated data and real world data

a blind source separation technique using second order statistics separation of sources consists of recovering a set of signals of which only instantaneous linear mixtures are observed in many situations no a priori information on the mixing matrix is available the linear mixture should be blindly processed this typically occurs in narrowband array processing applications when the array manifold is unknown or distorted this paper introduces a new source separation technique exploiting the time coherence of the source signals in contrast with other previously reported techniques the proposed approach relies only on stationary second order statistics that are based on a joint diagonalization of a set of covariance matrices asymptotic performance analysis of this method is carried out some numerical simulations are provided to illustrate the effectiveness of the proposed method

an expectation maximization method for spatio temporal blind source separation using an ar mog source model in this paper we develop a maximum likelihood ml spatio temporal blind source separation bss algorithm where the temporal dependencies are explained by assuming that each source is an autoregressive ar process and the distribution of the associated independent identically distributed i i d innovations process is described using a mixture of gaussians unlike most ml methods the proposed algorithm takes into account both spatial and temporal information optimization is performed using the expectation maximization em method the source model is adapted to maximize the likelihood and the update equations have a simple analytical form the proposed method which we refer to as autoregressive mixture of gaussians ar mog outperforms nine other methods for artificial mixtures of real audio we also show results for using ar mog to extract the fetal cardiac signal from real magnetocardiographic mcg data

biorthogonal vectors sesquilinear forms and some physical operators continuing the analysis undertaken in previous articles we discuss some features of non self adjoint operators and sesquilinear forms which are defined starting from two biorthogonal families of vectors like the so called generalized riesz systems enjoying certain properties in particular we discuss what happens when they forms two d quasi bases

markovian source separation a maximum likelihood ml approach is used to separate the instantaneous mixtures of temporally correlated independent sources with neither preliminary transformation nor a priori assumption about the probability distribution of the sources a markov model is used to represent the joint probability density of successive samples of each source the joint probability density functions are estimated from the observations using a kernel method for the special case of autoregressive models the theoretical performance of the algorithm is computed and compared with the performance of second order algorithms and i i d based separation algorithms

blind source separation by sparse decomposition in a signal dictionary the blind source separation problem is to extract the underlying source signals from a set of linear mixtures where the mixing matrix is unknown this situation is common in acoustics radio medical signal and image processing hyperspectral imaging and other areas we suggest a two stage separation process a priori selection of a possibly overcomplete signal dictionary for instance a wavelet frame or a learned dictionary in which the sources are assumed to be sparsely representable followed by unmixing the sources by exploiting the their sparse representability we consider the general case of more sources than mixtures but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques

cubature kalman filters in this paper we present a new nonlinear filter for high dimensional state estimation which we have named the cubature kalman filter ckf the heart of the ckf is a spherical radial cubature rule which makes it possible to numerically compute multivariate moment integrals encountered in the nonlinear bayesian filter specifically we derive a third degree spherical radial cubature rule that provides a set of cubature points scaling linearly with the state vector dimension the ckf may therefore provide a systematic solution for high dimensional nonlinear filtering problems the paper also includes the derivation of a square root version of the ckf for improved numerical stability the ckf is tested experimentally in two nonlinear state estimation problems in the first problem the proposed cubature rule is used to compute the second order statistics of a nonlinearly transformed gaussian random variable the second problem addresses the use of the ckf for tracking a maneuvering aircraft the results of both experiments demonstrate the improved performance of the ckf over conventional nonlinear filters

identification and estimation of non gaussian structural vector autoregressions abstract conventional structural vector autoregressive svar models with gaussian errors are not identified and additional identifying restrictions are needed in applied work we show that the gaussian case is an exception in that a svar model whose error vector consists of independent non gaussian components is without any additional restrictions identified and leads to essentially unique impulse responses building upon this result we introduce an identification scheme under which the maximum likelihood estimator of the parameters of the non gaussian svar model is consistent and asymptotically normally distributed as a consequence additional economic identifying restrictions can be tested in an empirical application we find a negative impact of a contractionary monetary policy shock on financial markets and clearly reject the commonly employed recursive identifying restrictions

wiener chaos expansions and numerical solutions of randomly forced equations of fluid mechanics in this paper we propose a numerical method based on wiener chaos expansion and apply it to solve the stochastic burgers and navier stokes equations driven by brownian motion the main advantage of the wiener chaos approach is that it allows for the separation of random and deterministic effects in a rigorous and effective manner the separation principle effectively reduces a stochastic equation to its associated propagator a system of deterministic equations for the coefficients of the wiener chaos expansion simple formulas for statistical moments of the stochastic solution are presented these formulas only involve the solutions of the propagator we demonstrate that for short time solutions the numerical methods based on the wiener chaos expansion are more efficient and accurate than those based on the monte carlo simulations

high degree cubature kalman filter the cubature kalman filter ckf which is based on the third degree spherical radial cubature rule is numerically more stable than the unscented kalman filter ukf but less accurate than the gauss hermite quadrature filter ghqf to improve the performance of the ckf a new class of ckfs with arbitrary degrees of accuracy in computing the spherical and radial integrals is proposed the third degree ckf is a special case of the class the high degree ckfs of the class can achieve the accuracy and stability performances close to those of the ghqf but at lower computational cost a numerical integration problem and a target tracking problem are utilized to demonstrate the necessity of using the high degree cubature rules to improve the performance the target tracking simulation shows that the fifth degree ckf can achieve higher accuracy than the extended kalman filter the ukf the third degree ckf and the particle filter and is computationally much more efficient than the ghqf

a coordinate descent algorithm for complex joint diagonalization under hermitian and transpose congruences abstract this paper deals with the problem of joint complexmatrix diagonalization by considering both the hermitian andtranspose congruences we address the general case where thesearched diagonalizing matrix is apriori nonunitary based onthe minimization of a quadratic criterion we propose a ex ible algorithm in the sense that it allows to directly consider arectangular diagonalizing matrix and to take into considerationboth the hermitian and transpose congruences within the sameframework the proposed algorithm is a coordinate descentalgorithm that is based on an approximate criterion leading to theanalytical derivation of the minima arguments computer simu lations are drawn to illustrate the usefulness and performancesof the algorithm and a comparison to state of the art algorithmsis presented finally an application to independent componentanalysis based on fourth order statistics is also presented index terms blind source separation complex symmetric ma trices hermitian matrices independent component analysis jointdiagonalization

a multiscale framework for blind separation of linearly mixed signals we consider the problem of blind separation of unknown source signals or images from a given set of their linear mixtures it was discovered recently that exploiting the sparsity of sources and their mixtures once they are projected onto a proper space of sparse representation improves the quality of separation in this study we take advantage of the properties of multiscale transforms such as wavelet packets to decompose signals into sets of local features with various degrees of sparsity we then study how the separation error is affected by the sparsity of decomposition coefficients and by the misfit between the probabilistic model of these coefficients and their actual distribution our error estimator based on the taylor expansion of the quasi ml function is used in selection of the best subsets of coefficients and utilized in turn in further separation the performance of the algorithm is evaluated by using noise free and noisy data experiments with simulated signals musical sounds and images demonstrate significant improvement of separation quality over previously reported results

faster independent component analysis by preconditioning with hessian approximations independent component analysis ica is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences in its classic form ica relies on modeling the data as linear mixtures of nongaussian independent sources the maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data this problem has been addressed by resorting to quasi newton methods which rely on sparse approximations of the hessian of the log likelihood however those approximations are not accurate when the ica model does not hold exactly as is often the case for real datasets we propose a new algorithm dubbed picard which makes use of sparse approximate hessians only as a preconditioner to the l bfgs algorithm refining the hessian approximation from a memory of the past iterates extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique especially on real data

statistical inference for independent component analysis application to structural var models abstract the well known problem of non identifiability of structural var models disappears if the structural shocks are independent and if at most one of them is gaussian in that case the relevant estimation technique is the independent component analysis ica since the introduction of ica by comon 1994 various semi parametric estimation methods have been proposed for orthogonalizing the error terms these methods include pseudo maximum likelihood pml approaches and recursive pml the aim of our paper is to derive the asymptotic properties of the pml approaches in particular to study their consistency we conduct monte carlo studies exploring the relative performances of these methods finally an application based on real data shows that structural var models can be estimated without additional identification restrictions in the non gaussian case and that the usual restrictions can be tested

mixture kalman filters in treating dynamic systems sequential monte carlo methods use discrete samples to represent a complicated probability distribution and use rejection sampling importance sampling and weighted resampling to complete the on line filtering task we propose a special sequential monte carlo method the mixture kalman filter which uses a random mixture of the gaussian distributions to approximate a target distribution it is designed for on line estimation and prediction of conditional and partial conditional dynamic linear models which are themselves a class of widely used non linear systems and also serve to approximate many others compared with a few available filtering methods including monte carlo methods the gain in efficiency that is provided by the mixture kalman filter can be very substantial another contribution of the paper is the formulation of many non linear systems into conditional or partial conditional linear form to which the mixture kalman filter can be applied examples in target tracking and digital communications are given to demonstrate the procedures proposed