blind separation of positive sources by globally convergent gradient search the instantaneous noise free linear mixing model in independent component analysis is largely a solved problem under the usual assumption of independent nongaussian sources and full column rank mixing matrix however with some prior information on the sources like positivity new analysis and perhaps simplified solution methods may yet become possible in this letter we consider the task of independent component analysis when the independent sources are known to be nonnegative and well grounded which means that they have a nonzero pdf in the region of zero it can be shown that in this case the solution method is basically very simple an orthogonal rotation of the whitened observation vector into nonnegative outputs will give a positive permutation of the original sources we propose a cost function whose minimum coincides with nonnegativity and derive the gradient algorithm under the whitening constraint under which the separating matrix is orthogonal we further prove that in the stiefel manifold of orthogonal matrices the cost function is a lyapunov function for the matrix gradient flow implying global convergence thus this algorithm is guaranteed to find the nonnegative well grounded independent sources the analysis is complemented by a numerical simulation which illustrates the algorithm

uniform decay for the unilateral problem associated to the kirchhoff type wave equations with nonlinear boundary damping in this article we study the existence of solutions and the uniform decay rate of the energy for nonlinear unilateral problem associated to the kirchhoff type wave equations with nonlinear boundary damping to obtain the existence of solutions we make use of faedo galerkin s approximation and also to show the uniform stabilization we use the perturbed energy method

blind source separation with nonlinear autocorrelation and non gaussianity blind source separation bss is a problem that is often encountered in many applications such as biomedical signal processing and analysis speech and image processing wireless telecommunication systems data mining sonar radar enhancement etc one often solves the bss problem by using the statistical properties of original sources e g non gaussianity or time structure information nevertheless real life mixtures are likely to contain both non gaussianity and time structure information sources rendering the algorithms using only one statistical property fail in this paper we address the bss problem when source signals have non gaussianity and temporal structure with nonlinear autocorrelation based on the two statistical characteristics of sources we develop an objective function maximizing the objective function we propose a gradient ascent source separation algorithm furthermore we give some mathematical properties for the algorithm computer simulations for sources with square temporal autocorrelation and non gaussianity illustrate the efficiency of the proposed approach

learning overcomplete representations in an overcomplete basis the number of basis vectors is greater than the dimensionality of the input and the representation of an input is not a unique combination of basis vectors overcomplete representations have been advocated because they have greater robustness in the presence of noise can be sparser and can have greater flexibility in matching structure in the data overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex previous work has focused on finding the best representation of a signal using a fixed overcomplete basis or dictionary we present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data we show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency this can be viewed as a generalization of the technique of independent component analysis and provides a method for bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures

bi iterative algorithm for extracting independent components from array signals it is well known that the complex valued nonunitary joint diagonalization nujd problem in blind source separation bss can be linked to the trilinear models in this paper by exploring the special structure of the nujd problem in bss we introduce a novel symmetric tri quadratic cost function and then derive a simultaneous bi iterative algorithm s bia for solving the nujd problem finally three experiments are conducted in order to show the comparatively good performance of the proposed bss algorithm

blind source separation by nonstationarity of variance a cumulant based approach blind separation of source signals usually relies either on the nongaussianity of the signals or on their linear autocorrelations a third approach was introduced by matsuoka et al 1995 who showed that source separation can be performed by using the nonstationarity of the signals in particular the nonstationarity of their variances in this paper we show how to interpret the nonstationarity due to a smoothly changing variance in terms of higher order cross cumulants this is based on the time correlation of the squares energies of the signals and leads to a simple optimization criterion using this criterion we construct a fixed point algorithm that is computationally very efficient

equivariant adaptive selective transmission in this paper we consider the problem of selective transmission the dual of the blind source separation task in which a set of independent source signals are adaptively premixed prior to a nondispersive physical mixing process so that each source can be independently monitored in the far field following similar procedures for information theoretic blind source separation we derive a stochastic gradient algorithm for iteratively estimating the premixing matrix in the selective transmission problem and through a simple modification we obtain a second algorithm whose performance is equivariant with respect to the channel s mixing characteristics the local stability conditions for the algorithms about any selective transmission solution are shown to be the same as those for similar source separation algorithms practical implementation issues are discussed including the estimation of the combined system matrix and the reordering and scaling of the received signals within the algorithm mean square error based selective transmission algorithms are also derived for performance comparison purposes simulations indicate the useful behavior of the premixing algorithms for selective transmission

a fixed point algorithm for blind source separation with nonlinear autocorrelation this paper addresses blind source separation bss problem when source signals have the temporal structure with nonlinear autocorrelation using the temporal characteristics of sources we develop an objective function based on the nonlinear autocorrelation of sources maximizing the objective function we propose a fixed point source separation algorithm furthermore we give some mathematical properties of the algorithm computer simulations for sources with square temporal autocorrelation and the real world applications in the analysis of the magnetoencephalographic recordings meg illustrate the efficiency of the proposed approach thus the presented bss algorithm which is based on the nonlinear measure of temporal autocorrelation provides a novel statistical property to perform bss

on uniform decay of the solution for a damped nonlinear coupled system of wave equations with nonlinear boundary damping and memory term in this paper we are concerned with the existence and energy decay of the solution to the initial boundary value problem for the nonlinear coupled wave equations with nonlinear boundary damping and memory term

joint independent subspace analysis using second order statistics this paper deals with a novel generalization of classical blind source separation bss in two directions first relaxing the constraint that the latent sources must be statistically independent this generalization is well known and sometimes termed independent subspace analysis isa second jointly analyzing several isa problems where the link is due to statistical dependence among corresponding sources in different mixtures when the data are one dimensional i e multiple classical bss problems this model known as independent vector analysis iva has already been studied in this paper we combine iva with isa and term this new model joint independent subspace analysis jisa we provide full performance analysis of jisa including closed form expressions for minimal mean square error mse fisher information and cramer rao lower bound in the separation of gaussian data the derived mse applies also for non gaussian data when only second order statistics are used we generalize previously known results on iva including its ability to uniquely resolve instantaneous mixtures of real gaussian stationary data and having the same arbitrary permutation at all mixtures numerical experiments validate our theoretical results and show the gain with respect to two competing approaches that either use a finer block partition or a different norm

long time asymptotic expansion for the damped semilinear wave equation the first initial boundary value problem is considered for the damped semilinear wave equation with the quadratic nonlinearity for small initial data and homogeneous boundary conditions its solution is constructed in the form of a series in the eigenfunctions of the laplace operator the long time asymptotic expansion is obtained which shows the nonlinear effects of amplitude and frequency multiplication the same results hold for the admissible initial data that are not small

one bit matching conjecture for independent component analysis the one bit matching conjecture for independent component analysis ica could be understood from different perspectives but is basically stated as all the sources can be separated as long as there is a one to one same sign correspondence between the kurtosis signs of all source probability density functions pdf s and the kurtosis signs of all model pdf s xu cheung amari 1998a this conjecture has been widely believed in the ica community and implicitly supported by many ica studies such as the extended infomax lee girolami sejnowski 1999 and the soft switching algorithm welling weber 2001 however there is no mathematical proof to confirm the conjecture theoretically in this article only skewness and kurtosis are considered and such a mathematical proof is given under the assumption that the skewness of the model densities vanishes moreover empirical experiments are demonstrated on the robustness of the conjecture as the vanishing skewness assumption breaks as a by product we also show that the kurtosis maximization criterion moreau macchi 1996 is actually a special case of the minimum mutual information criterion for ica

blind separation of independent sources for virtually any source probability density function the blind source separation bss problem consists of the recovery of a set of statistically independent source signals from a set of measurements that are mixtures of the sources when nothing is known about the sources and the mixture structure in the bss scenario of two noiseless real valued instantaneous linear mixtures of two sources an approximate maximum likelihood ml approach has been suggested in the literature which is only valid under certain constraints on the probability density function pdf of the sources in the present paper the expression for this ml estimator is reviewed and generalized to include virtually any source distribution an intuitive geometrical interpretation of the new estimator is also given in terms of the scatter plots of the signals involved an asymptotic performance analysis is then carried out yielding a closed form expression for the estimator asymptotic pdf simulations illustrate the behavior of the suggested estimator and show the accuracy of the asymptotic analysis in addition an extension of the method to the general bss scenario of more than two sources and two sensors is successfully implemented

a nonnegative pca algorithm for independent component analysis we consider the task of independent component analysis when the independent sources are known to be nonnegative and well grounded so that they have a nonzero probability density function pdf in the region of zero we propose the use of a nonnegative principal component analysis nonnegative pca algorithm which is a special case of the nonlinear pca algorithm but with a rectification nonlinearity and we conjecture that this algorithm will find such nonnegative well grounded independent sources under reasonable initial conditions while the algorithm has proved difficult to analyze in the general case we give some analytical results that are consistent with this conjecture and some numerical simulations that illustrate its operation

adaptive blind source separation for virtually any source probability density function blind source separation bss aims to recover a set of statistically independent source signals from a set of linear mixtures of the same sources in the noiseless real mixture two source two sensor scenario once the observations are whitened decorrelated and normalized only a givens rotation matrix remains to be identified in order to achieve the source separation in this paper an adaptive estimator of the angle that characterizes such a rotation is derived it is shown to converge to a stable valid separation solution with the only condition that the sum of source kurtosis be distinct from zero an asymptotic performance analysis is carried out resulting in a closed form expression for the asymptotic probability density function of the proposed estimator it is shown how the estimator can be incorporated into a complete adaptive source separation system by combining it with an adaptive prewhitening strategy and how it can be useful in a general bss scenario of more than two signals by means of a pairwise approach a variety of simulations assess the accuracy of the asymptotic results display the properties of the estimator such as its robust fast convergence and compare this on line bss implementation with other adaptive bss procedures

a constrained em algorithm for independent component analysis we introduce a novel way of performing independent component analysis using a constrained version of the expectation maximization em algorithm the source distributions are modeled as d one dimensional mixtures of gaussians the observed data are modeled as linear mixtures of the sources with additive isotropic noise this generative model is fit to the data using constrained em the simpler soft switching approach is introduced which uses only one parameter to decide on the sub or supergaussian nature of the sources we explain how our approach relates to independent factor analysis

hybrid linear and nonlinear complexity pursuit for blind source separation blind source separation bss is an increasingly popular data analysis technique with many applications several methods for bss using the statistical properties of original sources have been proposed for a famous case non gaussianity this leads to independent component analysis ica in this paper we propose a hybrid bss method based on linear and nonlinear complexity pursuit which combines three statistical properties of source signals non gaussianity linear predictability and nonlinear predictability a gradient learning algorithm is presented by minimizing a loss function simulations verify the efficient implementation of the proposed method

preconditioned stochastic gradient descent stochastic gradient descent sgd still is the workhorse for many practical problems however it converges slow and can be difficult to tune it is possible to precondition sgd to accelerate its convergence remarkably but many attempts in this direction either aim at solving specialized problems or result in significantly more complicated methods than sgd this paper proposes a new method to adaptively estimate a preconditioner such that the amplitudes of perturbations of preconditioned stochastic gradient match that of the perturbations of parameters to be optimized in a way comparable to newton method for deterministic optimization unlike the preconditioners based on secant equation fitting as done in deterministic quasi newton methods which assume positive definite hessian and approximate its inverse the new preconditioner works equally well for both convex and nonconvex optimizations with exact or noisy gradients when stochastic gradient is used it can naturally damp the gradient noise to stabilize sgd efficient preconditioner estimation methods are developed and with reasonable simplifications they are applicable to large scale problems experimental results demonstrate that equipped with the new preconditioner without any tuning effort preconditioned sgd can efficiently solve many challenging problems like the training of a deep neural network or a recurrent neural network requiring extremely long term memories

blind source separation using temporal predictability a measure of temporal predictability is defined and used to separate linear mixtures of signals given any set of statistically independent source signals it is conjectured here that a linear mixture of those signals has the following property the temporal predictability of any signal mixture is less than or equal to that of any of its component source signals it is shown that this property can be used to recover source signals from a set of linear mixtures of those signals by finding an un mixing matrix that maximizes a measure of temporal predictability for each recovered signal this matrix is obtained as the solution to a generalized eigenvalue problem such problems have scaling characteristics of o n3 where n is the number of signal mixtures in contrast to independent component analysis the temporal predictability method requires minimal assumptions regarding the probability density functions of source signals it is demonstrated that the method can separate signal mixtures in which each mixture is a linear combination of source signals with supergaussian subgaussian and gaussian probability density functions and on mixtures of voices and music

independent vector analysis identification conditions and performance bounds recently an extension of independent component analysis ica from one to multiple datasets termed independent vector analysis iva has been a subject of significant research interest iva has also been shown to be a generalization of hotelling s canonical correlation analysis in this paper we provide the identification conditions for a general iva formulation which accounts for linear nonlinear and sample to sample dependencies the identification conditions are a generalization of previous results for ica and for iva when samples are independently and identically distributed furthermore a principal aim of iva is identification of dependent sources between datasets thus we provide additional conditions for when the arbitrary ordering of the estimated sources can be common across datasets performance bounds in terms of the cramer rao lower bound are also provided for demixing matrices and interference to source ratio the performance of two iva algorithms are compared to the theoretical bounds

existence uniqueness and asymptotic behavior for solutions of the nonlinear beam equation a mathematical model for the transverse deflection of an extensible beam of length l whose ends are held at fixed distance apart is equation 2 u t 2 4 u x 4 0 l u 2 t d 2 u x 2 0 which has been proposed by woinowsky and krieger 18 where is a positive constant is a constant not necessarily positive and the nonlinear term represents the change in the tension of the beam due to its extensibility

on coupled wave equation of kirchhoff type with nonlinear boundary damping and memory term in this paper we prove the existence of the solution to the mixed problem for coupled wave equation of kirchhoff type with nonlinear boundary damping and memory term moreover we discuss the uniform decay of the solution

relative optimization for blind deconvolution we propose a relative optimization framework for quasi maximum likelihood qml blind deconvolution and the relative newton method as its particular instance special hessian structure allows fast newton system construction and solution resulting in a fast convergent algorithm with iteration complexity comparable to that of gradient methods we also propose the use of rational infinite impulse response iir restoration kernels which constitute a richer family of filters than the traditionally used finite impulse response fir kernels we discuss different choices of nonlinear functions that are suitable for deconvolution of super and sub gaussian sources and formulate the conditions under which the qml estimation is stable simulation results demonstrate the efficiency of the proposed methods

blind separation of signals with mixed kurtosis signs using threshold activation functions a parameterized activation function in the form of an adaptive threshold for a single layer neural network which separates a mixture of signals with any distribution except for gaussian is introduced this activation function is particularly simple to implement since it neither uses hyperbolic nor polynomial functions unlike most other nonlinear functions used for blind separation for some specific distributions the stable region of the threshold parameter is derived and optimal values for best separation performance are given if the threshold parameter is made adaptive during the separation process the successful separation of signals whose distribution is unknown is demonstrated and compared against other known methods

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

blind signal separation using steepest descent method a method that significantly improves the convergence rate of the gradient based blind signal separation bss algorithm for convolutive mixtures is proposed the proposed approach is based on the steepest descent algorithm suitable for constrained bss problems where the constraints are included to ease the permutation effects associated with the convolutive mixtures in addition the method is realized using a modified golden search method plus parabolic interpolation and this allows the optimum step size to be determined with only a few calculations of the cost function evaluation of the proposed procedure in simulated environments and in a real room environment shows that the proposed method results in significantly faster convergence for the bss when compared with a fixed step size gradient based algorithm in addition for blind signal extraction where only a main speech source is desired a combined scheme consisting of the proposed bss and a postprocessor such as an adaptive noise canceller offers impressive noise suppression levels while maintaining low target signal distortion levels

a generalized contrast function and stability analysis for overdetermined blind separation of instantaneous mixtures in this letter the problem of blind separation of n independent sources from their m linear instantaneous mixtures is considered first a generalized contrast function is defined as a valuable extension of the existing classical and nonsymmetrical contrast functions it is applicable to the overdetermined blind separation m n with an unknown number of sources because not only independent components but also redundant ones are allowed in the outputs of a separation system second a natural gradient learning algorithm developed primarily for the complete case m n is shown to work as well with an n m or m m separating matrix for each optimizes a certain mutual information contrast function finally we present stability analysis for a newly proposed generalized orthogonal natural gradient algorithm which can perform the overdetermined blind separation when n is unknown obtaining an expectable result that its local stability conditions are slightly stricter than those of the conventional natural gradient algorithm using an invertible mixing matrix m n

blind separation of instantaneous mixtures of nonstationary sources most source separation algorithms are based on a model of stationary sources however it is a simple matter to take advantage of possible nonstationarities of the sources to achieve separation this paper develops novel approaches in this direction based on the principles of maximum likelihood and minimum mutual information these principles are exploited by efficient algorithms in both the off line case via a new joint diagonalization procedure and in the on line case via a newton like procedure some experiments showing the good performance of our algorithms and evidencing an interesting feature of our methods are presented their ability to achieve a kind of super efficiency the paper concludes with a discussion contrasting separating methods for non gaussian and nonstationary models and emphasizing that as a matter of fact what makes the algorithms work is strictly speaking not the nonstationarity itself but rather the property that each realization of the source signals has a time varying envelope

independent component analysis over galois fields of prime order we consider the framework of independent component analysis ica for the case where the independent sources and their linear mixtures all reside in a galois field of prime order p similarities and differences from the classical ica framework over the real field are explored we show that a necessary and sufficient identifiability condition is that none of the sources should have a uniform distribution we also show that pairwise independence of the mixtures implies their full mutual independence namely a nonmixing condition in the binary p 2 and ternary p 3 cases but not necessarily in higher order p 3 cases we propose two different iterative separation or identification algorithms one is based on sequential identification of the smallest entropy linear combinations of the mixtures and is shown to be equivariant with respect to the mixing matrix the other is based on sequential minimization of the pairwise mutual information measures we provide some basic performance analysis for the binary p 2 case supplemented by simulation results for higher orders demonstrating advantages and disadvantages of the proposed separation approaches

fast nonlinear autocorrelation algorithm for source separation independent component analysis ica and blind source separation bss methods have been used for pattern recognition problems it is well known that ica and bss depend on the statistical properties of original sources or components such as non gaussianity in the paper using a statistical property nonlinear autocorrelation and maximizing the nonlinear autocorrelation of source signals we propose a fast fixed point algorithm for bss we study its convergence property and show that its convergence speed is at least quadratic simulations by the artificial signals and the real world applications verify the efficient implementation of the proposed method

algorithms for complex ml ica and their stability analysis using wirtinger calculus we derive a class of algorithms for independent component analysis ica based on maximum likelihood ml estimation and perform stability analysis of natural gradient ml ica with and without the constraint for unitary demixing matrix in the process we demonstrate how wirtinger calculus facilitates derivations and most importantly performing second order analysis in the complex domain and eliminates the need for making simplifying assumptions we derive natural gradient complex ml ica update rule and its variant with a unitary constraint as well as a newton algorithm for better convergence behavior the conditions for local stability are derived and studied using a generalized gaussian density ggd source model when the sources are circular and non gaussian we show analytically that both the ml and ml unitary ica update rules converge to the inverse of mixing matrix subject to a phase shift when the sources are noncircular and non gaussian we show that the nonunitary ml ica update rule is more stable than the ml unitary ica update rule when the sources are noncircular gaussians both update rules are stable only when the sources have distinct noncircularity indices simulation results are given to support these results

statistical dynamics of on line independent component analysis the learning dynamics of on line independent component analysis is analysed in the limit of large data dimension we study a simple hebbian learning algorithm that can be used to separate out a small number of non gaussian components from a high dimensional data set the de mixing matrix parameters are confined to a stiefel manifold of tall orthogonal matrices and we introduce a natural gradient variant of the algorithm which is appropriate to learning on this manifold for large input dimension the parameter trajectory of both algorithms passes through a sequence of unstable fixed points each described by a diffusion process in a polynomial potential choosing the learning rate too large increases the escape time from each of these fixed points effectively trapping the learning in a sub optimal state in order to avoid these trapping states a very low learning rate must be chosen during the learning transient resulting in learning time scales of o n2 or o n3 iterations where n is the data dimension escape from each sub optimal state results in a sequence of symmetry breaking events as the algorithm learns each source in turn this is in marked contrast to the learning dynamics displayed by related on line learning algorithms for multilayer neural networks and principal component analysis although the natural gradient variant of the algorithm has nice asymptotic convergence properties it has an equivalent transient dynamics to the standard hebbian algorithm

nonlinear pattern matching in trees tree pattern matching is a fundamental operation that is used in a number of programming tasks such as mechanical theorem proving term rewriting symbolic computation and nonprocedural programming languages in this paper we present new sequential algorithms for nonlinear pattern matching in trees our algorithm improves upon know tree pattern matching algorithms in important aspects such as time performance ease of integration with several reduction strategies and ability to avoid unnecessary computation steps on match attempts that fail the expected time complexity of our algorithm is linear in the sum of the sizes of the two trees

globally convergent blind source separation based on a multiuser kurtosis maximization criterion we consider the problem of recovering blindly i e without the use of training sequences a number of independent and identically distributed source user signals that are transmitted simultaneously through a linear instantaneous mixing channel the received signals are hence corrupted by interuser interference iui and we can model them as the outputs of a linear multiple input multiple output mimo memoryless system assuming the transmitted signals to be mutually independent i i d and to share the same non gaussian distribution a set of necessary and sufficient conditions for the perfect blind recovery up to scalar phase ambiguities of all the signals exists and involves the kurtosis as well as the covariance of the output signals we focus on a straightforward blind constrained criterion stemming from these conditions from this criterion we derive an adaptive algorithm for blind source separation which we call the multiuser kurtosis muk algorithm at each iteration the algorithm combines a stochastic gradient update and a gram schmidt orthogonalization procedure in order to satisfy the criterion s whiteness constraints a performance analysis of its stationary points reveals that the muk algorithm is free of any stable undesired local stationary points for any number of sources hence it is globally convergent to a setting that recovers them all

global existence and blow up of solutions for some hyperbolic systems with damping and source terms abstract this paper deals with the global existence and blow up of solutions to some nonlinear hyperbolic systems with damping and source terms in a bounded domain by using the potential well method we obtain the global existence moreover for the problem with linear damping terms blow up of solutions is considered and some estimates for the lifespan of solutions are given

waveform preserving blind estimation of multiple independent sources the problem of blind estimation of source signals is to estimate the source signals without knowing the characteristics of the transmission channel it is shown that the minimum variance unbiased estimates can be obtained if and only if the transmission channel can be identified blindly it is shown that the channel can be blindly identified if and only if there is not more than one gaussian source this condition suggests that waveform preserving blind estimation can be achieved over a wide range of signal processing applications including those cases in which the source signals have identical nongaussian distributions the constructive proof of the necessary and sufficient condition serves as a foundation for the development of waveform preserving blind signal estimation algorithms examples are presented to demonstrate the applications of the theoretical results

estimating functions of independent component analysis for temporally correlated signals this article studies a general theory of estimating functions of independent component analysis when the independent source signals are temporarily correlated estimating functions are used for deriving both batch and on line learning algorithms and they are applicable to blind cases where spatial and temporal probability structures of the sources are unknown most algorithms proposed so far can be analyzed in the framework of estimating functions an admissible class of estimating functions is derived and related efficient on line learning algorithms are introduced we analyze dynamical stability and statistical efficiency of these algorithms different from the independently and identically distributed case the algorithms work even when only the second order moments are used the method of simultaneous diagonalization of cross covariance matrices is also studied from the point of view of estimating functions

patterns and pattern matching in trees an analysis this paper presents general results on the probabilities of occurrences of patterns in trees which serve to analyze a commonly used pattern matching algorithm on trees it appears that for a wide class of statistics on trees pattern matching has a linear expected time complexity in contrast to a quadratic worst case behaviour the methods used are both combinatorial and analytic and prove useful in the evaluation of a wide class of tree algorithms

fast and robust fixed point algorithms for independent component analysis independent component analysis ica is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible we use a combination of two different approaches for linear ica comon s information theoretic approach and the projection pursuit approach using maximum entropy approximations of differential entropy we introduce a family of new contrast functions for ica these contrast functions enable both the estimation of the whole decomposition by minimizing mutual information and estimation of individual independent components as projection pursuit directions the statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model and it is shown how to choose contrast functions that are robust and or of minimum variance finally we introduce simple fixed point algorithms for practical optimization of the contrast functions

penalty function based joint diagonalization approach for convolutive blind separation of nonstationary sources a new approach for convolutive blind source separation bss by explicitly exploiting the second order nonstationarity of signals and operating in the frequency domain is proposed the algorithm accommodates a penalty function within the cross power spectrum based cost function and thereby converts the separation problem into a joint diagonalization problem with unconstrained optimization this leads to a new member of the family of joint diagonalization criteria and a modification of the search direction of the gradient based descent algorithm using this approach not only can the degenerate solution induced by a null unmixing matrix and the effect of large errors within the elements of covariance matrices at low frequency bins be automatically removed but in addition a unifying view to joint diagonalization with unitary or nonunitary constraint is provided numerical experiments are presented to verify the performance of the new method which show that a suitable penalty function may lead the algorithm to a faster convergence and a better performance for the separation of convolved speech signals in particular in terms of shape preservation and amplitude ambiguity reduction as compared with the conventional second order based algorithms for convolutive mixtures that exploit signal nonstationarity

complexity pursuit separating interesting components from time series a generalization of projection pursuit for time series that is signals with time structure is introduced the goal is to find projections of time series that have interesting structure defined using criteria related to kolmogoroff complexity or coding length interesting signals are those that can be coded with a short code length we derive a simple approximation of coding length that takes into account both the nongaussianity and the autocorrelations of the time series also we derive a simple algorithm for its approximative optimization the resulting method is closely related to blind separation of nongaussian time dependent source signals

a singular perturbation problem for nonlinear damped hyperbolic equations on considere des equations hyperboliques non lineaires amorties u tt au t au a 2 u g u 0 ou a est un operateur positif et g est la derivee de gateaux d une fonctionnelle convexe on etudie le comportement asymptotique des solutions et la convergence des solutions fortes quand o

adaptive modal identification of structures with equivariant adaptive separation via independence approach abstract an efficient output only blind source separation bss method was recently introduced for the modal identification of structures bss procedures recover a set of independent sources from their unknown linear mixtures when only mixtures are observed batch data is required for the separation in traditional blind source separation methods these algorithms are however unfavorable as some sets of data are observed one after another in this paper an adaptive blind source separation technique equivariant adaptive separation via independence easi is introduced to overcome the mentioned disadvantage within the structures the easi algorithm is beneficial as it can provide solutions to real time problems while also update the un mixing matrix for each step easi not only avoids increases in size of the relevant matrices and vectors but also decreases the analysis time a synthetic example and a benchmark structure have been used in this paper to better investigate the efficiency of the proposed method the simulation results demonstrate the effectiveness of the easi algorithm in on line identification of modal parameters of structures

existence of periodic solutions for nonlinear differential equations abstract in this paper we shall study the existence of a periodic solution to the nonlinear differential equationxtt axt a xt a ax b t x f t x t 0 in some complex hilbert space using duality and variational methods

analysis of sparse representation and blind source separation in this letter we analyze a two stage cluster then l1 optimization approach for sparse representation of a data matrix which is also a promising approach for blind source separation bss in which fewer sensors than sources are present first sparse representation factorization of a data matrix is discussed for a given overcomplete basis matrix the corresponding sparse solution coefficient matrix with minimum l1 norm is unique with probability one which can be obtained using a standard linear programming algorithm the equivalence of the l1 norm solution and the l0 norm solution is also analyzed according to a probabilistic framework if the obtained l1 norm solution is sufficiently sparse then it is equal to the l0 norm solution with a high probability furthermore the l1 norm solution is robust to noise but the l0 norm solution is not showing that the l1 norm is a good sparsity measure these results can be used as a recoverability analysis of bss as discussed the basis matrix in this article is estimated using a clustering algorithm followed by normalization in which the matrix columns are the cluster centers of normalized data column vectors zibulevsky pearlmutter boll and kisilev 2000 used this kind of two stage approach in underdetermined bss our recoverability analysis shows that this approach can deal with the situation in which the sources are overlapped to some degree in the analyzed domain and with the case in which the source number is unknown it is also robust to additive noise and estimation error in the mixing matrix finally four simulation examples and an eeg data analysis example are presented to illustrate the algorithm s utility and demonstrate its performance

global existence and stability for wave equation of kirchhoff type with memory condition at the boundary abstract we consider a nonlinear wave equation of kirchhoff type with memory condition at the boundary and we study the asymptotic behavior of the corresponding solutions we proved that the energy decay with the same rate of decay of the relaxation function that is the energy decays exponentially when the relaxation function decay exponentially and polynomially when the relaxation function decay polynomially

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

a linear non gaussian acyclic model for causal discovery in recent years several methods have been proposed for the discovery of causal structure from non experimental data such methods make various assumptions on the data generating process to facilitate its identification from purely observational data continuing this line of research we show how to discover the complete causal structure of continuous valued data under the assumptions that a the data generating process is linear b there are no unobserved confounders and c disturbance variables have non gaussian distributions of non zero variances the solution relies on the use of the statistical method known as independent component analysis and does not require any pre specified time ordering of the variables we provide a complete matlab package for performing this lingam analysis short for linear non gaussian acyclic model and demonstrate the effectiveness of the method using artificially generated data and real world data

a blind source separation technique using second order statistics separation of sources consists of recovering a set of signals of which only instantaneous linear mixtures are observed in many situations no a priori information on the mixing matrix is available the linear mixture should be blindly processed this typically occurs in narrowband array processing applications when the array manifold is unknown or distorted this paper introduces a new source separation technique exploiting the time coherence of the source signals in contrast with other previously reported techniques the proposed approach relies only on stationary second order statistics that are based on a joint diagonalization of a set of covariance matrices asymptotic performance analysis of this method is carried out some numerical simulations are provided to illustrate the effectiveness of the proposed method

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

sequential blind source separation based exclusively on second order statistics developed for a class of periodic signals a sequential algorithm for the blind separation of a class of periodic source signals is introduced in this paper the algorithm is based only on second order statistical information and exploits the assumption that the source signals have distinct periods separation is performed by sequentially converging to a solution which in effect diagonalizes the output covariance matrix constructed at a lag corresponding to the fundamental period of the source we select the one with the smallest period simulation results for synthetic signals and real electrocardiogram recordings show that the proposed algorithm has the ability to restore statistical independence and its performance is comparable to that of the equivariant adaptive source separation easi algorithm a benchmark high order statistics based sequential algorithm with similar computational complexity the proposed algorithm is also shown to mitigate the limitation that the easi algorithm can separate at most one gaussian distributed source furthermore the steady state performance of the proposed algorithm is compared with that of easi and the block based second order blind identification sobi method

a new constrained fixed point algorithm for ordering independent components independent component analysis ica aims to recover a set of unknown mutually independent components ics from their observed mixtures without knowledge of the mixing coefficients in the classical ica model there exists ics indeterminacy on permutation and dilation constrained ica is one of methods for solving this problem through introducing constraints into the classical ica model in this paper we first present a new constrained ica model which composed of three parts a maximum likelihood criterion as an objective function statistical measures as inequality constraints and the normalization of demixing matrix as equality constraints next we incorporate the new fixed point newfp algorithm into this constrained ica model to construct a new constrained fixed point algorithm computation simulations on synthesized signals and speech signals demonstrate that this combination both can eliminate ics indeterminacy to a certain extent and can provide better performance moreover comparison results with the existing algorithm verify the efficiency of our new algorithm furthermore and show that it is more simple to implement than the existing algorithm due to its advantage of not using the learning rate finally this new algorithm is also applied for the real world fetal ecg data experiment results further indicate the efficiency of the new constrained fixed point algorithm

the depth first processes of galton watson trees converge to the same brownian excursion in this paper we show a strong relation between the depth first processes associated to galton watson trees with finite variance conditioned by the total progeny the depth first walk the depth first queue process the height process a consequence is that these processes suitably normalized converge to the same brownian excursion this provides an alternative proof of aldous one of the convergence of the depth first walk to the brownian excursion which does not use the existence of a limit tree the methods that we introduce allow one to compute some functionals of trees or discrete excursions for example we compute the limit law of the process of the height of nodes with a given out degree and the process of the height of nodes root of a given subtree

blind source separation by sparse decomposition in a signal dictionary the blind source separation problem is to extract the underlying source signals from a set of linear mixtures where the mixing matrix is unknown this situation is common in acoustics radio medical signal and image processing hyperspectral imaging and other areas we suggest a two stage separation process a priori selection of a possibly overcomplete signal dictionary for instance a wavelet frame or a learned dictionary in which the sources are assumed to be sparsely representable followed by unmixing the sources by exploiting the their sparse representability we consider the general case of more sources than mixtures but also derive a more efficient algorithm in the case of a nonovercomplete dictionary and an equal numbers of sources and mixtures experiments with artificial signals and musical sounds demonstrate significantly better separation than other known techniques

independent factor analysis we introduce the independent factor analysis ifa method for recovering independent hidden sources from their observed mixtures ifa generalizes and unifies ordinary factor analysis fa principal component analysis pca and independent component analysis ica and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy ifa is a two step procedure in the first step the source densities mixing matrix and noise covariance are estimated from the observed data by maximum likelihood for this purpose we present an expectation maximization em algorithm which performs unsupervised learning of an associated probabilistic model of the mixing situation each source in our model is described by a mixture of gaussians thus all the probabilistic calculations can be performed analytically in the second step the sources are reconstructed from the observed data by an optimal nonlinear estimator a variational approximation of this algorithm is derived for cases with a large number of sources where the exact algorithm becomes intractable our ifa algorithm reduces to the one for ordinary fa when the sources become gaussian and to an em algorithm for pca in the zero noise limit we derive an additional em algorithm specifically for noiseless ifa this algorithm is shown to be superior to ica since it can learn arbitrary source densities from the data beyond blind separation ifa can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding

recognizable formal power series on trees trees are a very basic object in computer science they intervene in nearly a ly domain and they are studied for their own or used to represent conveniently a given situation there are at least three directions where investigations on trees themselves are motivated and this for different reasons first the notion of tree is the basis of algebraic semantics nivat 19 rosen 2 2 etc in this context the study of special languages of trees i e forests their classification and their behaviour under various types of transformations are of great importance arnold fl dauchet 8 j l ilin 1161 mongy 18 by essence work in this area is an extension of the algebraic theory of languages trees and ianguages are in fact directly related via the derivation trees of an algebraic grammar thatcher 25 a second topic heavily related to trees concerns dzra structures trees mainly binary trees and its variants constitute one of the most widely known data structures see e g hlnuth 15 the analysis of the worst cale expected or average running time t ehzvgrr nfi certain algorithms requires sometimes long and delicate computations flajolet lo kemp 14 flajolet and steyaert 12 finally trees occupy a distinguished place in the enumeration of graphs and maps both because of the simplicity of their structure and for the relationship between their el lcodings and aigebraic languages the nature of the enumerating series and especially the question whether they are algebraic or not is one of the central problems in this domain cori 7 chottin 4 we propose here a theory of formal power series on trees and present some of their basic properties together with various examples of applications which as we hope will show the interest of its development within the framework we just sketched a formal power series on trees is a function which gssociates a number to esch tree thus we could also have called them tree functions ii in analogy with the term word function used by several authors paz and salomaa zo cobham 5 as an equivalent denomination for formal power series on words the main goal of a formal power series is to count or to represent the result of some computation on

on a quasilinear wave equation with memory equation 1 l with a 0 and n 1 has its roots in a model for the small amplitude vibrations of a string in which the dependence of the tension on the deformation cannot be ignored see for example bernstein 2 carrier 3 41 dickey s and nishida 13 when a 0 1 1 may be used to describe the dynamics of an extensible string with fading memory this equation states that the dynamic equilibrium of a body depends not only on the present state of deformation but also on the previous history of the deformation gradient since the publication of the aforementioned papers equation 1 1 with a 0 has been the subject of intense scrutiny see for example arosio and spagnolo 11 lions 9 medeiros lo menzala ll nishihara 14 151 pohoiaev 16 and yamada 17 181 to name but a few among the facts thus far proved pohoiaev s results and their extension to the degenerate case of arosio and spagnolo are of particular significance they show that if the data satisfy an analytic type condition then solutions are global in time in spite of the abundant literature on the memoryless form of equation l l to our best knowledge equation 1 1 with a f 0 appears to be untouched our main results theorem 5 1 theorem 5 2 and theorem 5 4 will show that under appropriate assumptions on the kernel

new criteria for blind deconvolution of nonminimum phase systems channels a necessary and sufficient condition for blind deconvolution without observing the input of nonminimum phase linear time invariant systems channels is derived based on this condition several optimization criteria are proposed and their solution is shown to correspond to the desired response these criteria involve the computation only of second and fourth order moments implying a simple tap update procedure the proposed methods are universal in the sense that they do not impose any restrictions on the probability distribution of the unobserved input sequence it is shown that in several important cases e g when the additive noise is gaussian the proposed criteria are essentially unaffected

faster independent component analysis by preconditioning with hessian approximations independent component analysis ica is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences in its classic form ica relies on modeling the data as linear mixtures of nongaussian independent sources the maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data this problem has been addressed by resorting to quasi newton methods which rely on sparse approximations of the hessian of the log likelihood however those approximations are not accurate when the ica model does not hold exactly as is often the case for real datasets we propose a new algorithm dubbed picard which makes use of sparse approximate hessians only as a preconditioner to the l bfgs algorithm refining the hessian approximation from a memory of the past iterates extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique especially on real data

stochastic trapping in a solvable model of on line independent component analysis previous analytical studies of on line independent component analysis ica learning rules have focused on asymptotic stability and efficiency in practice the transient stages of learning are often more significant in determining the success of an algorithm this is demonstrated here with an analysis of a hebbian ica algorithm which can find a small number of nongaussian components given data composed of a linear mixture of independent source signals an idealized data model is considered in which the sources comprise a number of nongaussian and gaussian sources and a solution to the dynamics is obtained in the limit where the number of gaussian sources is infinite previous stability results are confirmed by expanding around optimal fixed points where a closed form solution to the learning dynamics is obtained however stochastic effects are shown to stabilize otherwise unstable suboptimal fixed points conditions required to destabilize one such fixed point are obtained for the case of a single nongaussian component indicating that the initial learning rate required to escape successfully is very low o n 2 where n is the data dimension resulting in very slow learning typically requiring o n3 iterations simulations confirm that this picture holds for a finite system

statistical inference for independent component analysis application to structural var models abstract the well known problem of non identifiability of structural var models disappears if the structural shocks are independent and if at most one of them is gaussian in that case the relevant estimation technique is the independent component analysis ica since the introduction of ica by comon 1994 various semi parametric estimation methods have been proposed for orthogonalizing the error terms these methods include pseudo maximum likelihood pml approaches and recursive pml the aim of our paper is to derive the asymptotic properties of the pml approaches in particular to study their consistency we conduct monte carlo studies exploring the relative performances of these methods finally an application based on real data shows that structural var models can be estimated without additional identification restrictions in the non gaussian case and that the usual restrictions can be tested

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing