pairwise likelihood ratios for estimation of non gaussian structural equation models we present new measures of the causal direction or direction of effect between two non gaussian random variables they are based on the likelihood ratio under the linear non gaussian acyclic model lingam we also develop simple first order approximations of the likelihood ratio and analyze them based on related cumulant based measures which can be shown to find the correct causal directions we show how to apply these measures to estimate lingam for more than two variables and even in the case of more variables than observations we further extend the method to cyclic and nonlinear models the proposed framework is statistically at least as good as existing ones in the cases of few data points or noisy data and it is computationally and conceptually very simple results on simulated fmri data indicate that the method may be useful in neuroimaging where the number of time points is typically quite small

blind separation of positive sources by globally convergent gradient search the instantaneous noise free linear mixing model in independent component analysis is largely a solved problem under the usual assumption of independent nongaussian sources and full column rank mixing matrix however with some prior information on the sources like positivity new analysis and perhaps simplified solution methods may yet become possible in this letter we consider the task of independent component analysis when the independent sources are known to be nonnegative and well grounded which means that they have a nonzero pdf in the region of zero it can be shown that in this case the solution method is basically very simple an orthogonal rotation of the whitened observation vector into nonnegative outputs will give a positive permutation of the original sources we propose a cost function whose minimum coincides with nonnegativity and derive the gradient algorithm under the whitening constraint under which the separating matrix is orthogonal we further prove that in the stiefel manifold of orthogonal matrices the cost function is a lyapunov function for the matrix gradient flow implying global convergence thus this algorithm is guaranteed to find the nonnegative well grounded independent sources the analysis is complemented by a numerical simulation which illustrates the algorithm

transmutations l bases and complete families of solutions of the stationary schrodinger equation in the plane an l basis associated to a linear second order ordinary differential operator l is an infinite sequence of functions k k 0 such that l k 0 for k 0 1 l k k k 1 k 2 for k 2 3 and all k satisfy certain prescribed initial conditions we study the transmutation operators related to l in terms of the transformation of powers of the independent variable x x0 k k 0 to the elements of the l basis and establish a precise form of the transmutation operator realizing this transformation we use this transmutation operator to establish a completeness of an infinite system of solutions of the stationary schrodinger equation from a certain class the system of solutions is obtained as an application of the theory of bicomplex pseudoanalytic functions and its completeness was a long sought result its use for constructing reproducing kernels and solving boundary and eigenvalue problems has been considered even without the required completeness justification the obtained result on the completeness opens the way for further development and application of the tools of pseudoanalytic function theory

learning overcomplete representations in an overcomplete basis the number of basis vectors is greater than the dimensionality of the input and the representation of an input is not a unique combination of basis vectors overcomplete representations have been advocated because they have greater robustness in the presence of noise can be sparser and can have greater flexibility in matching structure in the data overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex previous work has focused on finding the best representation of a signal using a fixed overcomplete basis or dictionary we present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data we show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency this can be viewed as a generalization of the technique of independent component analysis and provides a method for bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures

a non hybrid method for the pdf equations of turbulent flows on unstructured grids in probability density function pdf methods of turbulent flows the joint pdf of several flow variables is computed by numerically integrating a system of stochastic differential equations for lagrangian particles a set of parallel algorithms is proposed to provide an efficient solution of the pdf transport equation modeling the joint pdf of turbulent velocity frequency and concentration of a passive scalar in geometrically complex configurations in the vicinity of walls the flow is resolved by an elliptic relaxation technique down to the viscous sublayer explicitly modeling the high anisotropy and inhomogeneity of the low reynolds number wall region without damping or wall functions an unstructured eulerian grid is employed to extract eulerian statistics to solve for quantities represented at fixed locations of the domain i e the mean pressure and the elliptic relaxation tensor and to track particles all three aspects regarding the grid make use of the finite element method employing the simplest linear shapefunctions to model the small scale mixing of the transported scalar the interaction by exchange with the conditional mean iecm model is adopted an adaptive algorithm to compute the velocity conditioned scalar mean is proposed that homogenizes the statistical error over the sample space with no assumption on the shape of the underlying velocity pdf compared to other hybrid particle in cell approaches for the pdf equations the current methodology is consistent without the need for consistency conditions the algorithm is tested by computing the dispersion of passive scalars released from concentrated sources in two different turbulent flows the fully developed turbulent channel flow and a street canyon or cavity flow algorithmic details on estimating conditional and unconditional statistics particle tracking and particle number control are presented in detail relevant aspects of performance and parallelism on cache based shared memory machines are discussed

the hybrid method for the pdf equations of turbulent reactive flows consistency conditions and correction algorithms the hybrid method solves the modeled transport equation for the joint pdf of velocity turbulence frequency and compositions for turbulent reactive flows a finitevolume fv method is used to solve the mean conservation equations for mass momentum and energy and the mean equation of state and a particle method is used to solve the modeled pdf equation the method is completely consistent at the level of the governing equations solved by the fv and particle algorithms in this work the conditions to be fulfilled for full consistency at the numerical solution level are examined and the independent consistency conditions are identified then correction algorithms are developed to enforce these independent consistency conditions to achieve full consistency at the numerical solution level in addition a new formulation of the energy equation and the equation of state is developed which is both general and simple the hybrid method is applied to a non premixed piloted jet flame the numerical results show that the correction algorithms are completely successful in achieving consistency the convergence of the method is demonstrated and in particular it is shown that the bias error is dramatically reduced compared to that in previous pdf calculations in addition the results are shown to be in a good agreement with some earlier pdf calculations and also with the available experimental data because of the substantially reduced numerical error for given grid size and number of particles the present hybrid method represents a significant advance in the computational efficiency of particle mesh method for the solution of pdf equations c 2001 academic press

bi iterative algorithm for extracting independent components from array signals it is well known that the complex valued nonunitary joint diagonalization nujd problem in blind source separation bss can be linked to the trilinear models in this paper by exploring the special structure of the nujd problem in bss we introduce a novel symmetric tri quadratic cost function and then derive a simultaneous bi iterative algorithm s bia for solving the nujd problem finally three experiments are conducted in order to show the comparatively good performance of the proposed bss algorithm

on a relation of pseudoanalytic function theory to the two dimensional stationary schrodinger equation and taylor series in formal powers for its solutions we consider the real stationary two dimensional schrodinger equation with the aid of any of its particular solutions we construct a vekua equation possessing the following special property the real parts of its solutions are solutions of the original schrodinger equation and the imaginary parts are solutions of an associated schrodinger equation with a potential having the form of a potential obtained after the darboux transformation using bers theory of taylor series for pseudoanalytic functions we obtain a locally complete system of solutions of the original schrodinger equation which can be constructed explicitly for an ample class of schrodinger equations for example it is possible when the potential is a function of one cartesian spherical parabolic or elliptic variable we give some examples of application of the proposed procedure for obtaining a locally complete system of solutions of the schrodinger equation the procedure is algorithmically simple and can be implemented with the aid of a computer system of symbolic or numerical calculation

accurate and computationally efficient mixing models for the simulation of turbulent mixing with pdf methods abstract different simulation methods are applicable to study turbulent mixing when applying probability density function pdf methods turbulent transport and chemical reactions appear in closed form which is not the case in second moment closure methods rans moreover pdf methods provide the entire joint velocity scalar pdf instead of a limited set of moments in pdf methods however a mixing model is required to account for molecular diffusion in joint velocity scalar pdf methods mixing models should also account for the joint velocity scalar statistics which is often under appreciated in applications the interaction by exchange with the conditional mean iecm model accounts for these joint statistics but requires velocity conditional scalar means that are expensive to compute in spatially three dimensional settings in this work two alternative mixing models are presented that provide more accurate pdf predictions at reduced computational cost compared to the iecm model since no conditional moments have to be computed all models are tested for different mixing benchmark cases and their computational efficiencies are inspected thoroughly the benchmark cases involve statistically homogeneous and inhomogeneous settings dealing with three streams that are characterized by two passive scalars the inhomogeneous case clearly illustrates the importance of accounting for joint velocity scalar statistics in the mixing model failure to do so leads to significant errors in the resulting scalar means variances and other statistics

equivariant adaptive selective transmission in this paper we consider the problem of selective transmission the dual of the blind source separation task in which a set of independent source signals are adaptively premixed prior to a nondispersive physical mixing process so that each source can be independently monitored in the far field following similar procedures for information theoretic blind source separation we derive a stochastic gradient algorithm for iteratively estimating the premixing matrix in the selective transmission problem and through a simple modification we obtain a second algorithm whose performance is equivariant with respect to the channel s mixing characteristics the local stability conditions for the algorithms about any selective transmission solution are shown to be the same as those for similar source separation algorithms practical implementation issues are discussed including the estimation of the combined system matrix and the reordering and scaling of the received signals within the algorithm mean square error based selective transmission algorithms are also derived for performance comparison purposes simulations indicate the useful behavior of the premixing algorithms for selective transmission

an accurate time advancement algorithm for particle tracking we describe a particle position time advancement algorithm that is designed for use with several subgrid velocity reconstruction schemes used in les fdf methods and potentially in other applications these reconstruction schemes yield a subgrid velocity field with desirable divergence properties but also with discontinuities across cell faces therefore a conventional time advancement algorithm such as second order runge kutta rk2 does not perform as well as it does with a smooth velocity field the algorithm that we describe called multi step rk2 mrk2 builds upon rk2 by breaking up the time step into two or more substeps whenever a particle crosses one or more velocity discontinuities when used in conjunction with the parabolic edge reconstruction method mrk2 performs considerably better than rk2 both the final position of an advected particle and the final area of a 2d infinitesimal area element are second order accurate in time as opposed to first order accurate for rk2 furthermore mrk2 has the theoretical advantage that it better preserves the continuity of the mapping between initial and final particle positions

a solution algorithm for the fluid dynamic equations based on a stochastic model for molecular motion in this paper a stochastic model is presented to simulate the flow of gases which are not in thermodynamic equilibrium like in rarefied or micro situations for the interaction of a particle with others statistical moments of the local ensemble have to be evaluated but unlike in molecular dynamics simulations or dsmc no collisions between computational particles are considered in addition a novel integration technique allows for time steps independent of the stochastic time scale the stochastic model represents a fokker planck equation in the kinetic description which can be viewed as an approximation to the boltzmann equation this allows for a rigorous investigation of the relation between the new model and classical fluid and kinetic equations the fluid dynamic equations of navier stokes and fourier are fully recovered for small relaxation times while for larger values the new model extents into the kinetic regime numerical studies demonstrate that the stochastic model is consistent with navier stokes in that limit but also that the results become significantly different if the conditions for equilibrium are invalid the application to the knudsen paradox demonstrates the correctness and relevance of this development and comparisons with existing kinetic equations and standard solution algorithms reveal its advantages moreover results of a test case with geometrically complex boundaries are presented

an iterative inversion approach to blind source separation we present an iterative inversion ii approach to blind source separation bss it consists of a quasi newton method for the resolution of an estimating equation obtained from the implicit inversion of a robust estimate of the mixing system the resulting learning rule includes several existing algorithms for bss as particular cases giving them a novel and unified interpretation it also provides a justification of the cardoso and laheld 1996 step size normalization the ii method is first presented for instantaneous mixtures and then extended to the problem of blind separation of convolutive mixtures finally we derive the necessary and sufficient asymptotic stability conditions for both the instantaneous and convolutive methods to converge

compact cramer rao bound expression for independent component analysis despite of the increased interest in independent component analysis ica during the past two decades a simple closed form expression of the cramer rao bound crb for the demixing matrix estimation has not been established in the open literature in the present paper we fill this gap by deriving a simple closed form expression for the crb of the demixing matrix directly from its definition a simulation study comparing ica estimators with the crb is given

joint independent subspace analysis using second order statistics this paper deals with a novel generalization of classical blind source separation bss in two directions first relaxing the constraint that the latent sources must be statistically independent this generalization is well known and sometimes termed independent subspace analysis isa second jointly analyzing several isa problems where the link is due to statistical dependence among corresponding sources in different mixtures when the data are one dimensional i e multiple classical bss problems this model known as independent vector analysis iva has already been studied in this paper we combine iva with isa and term this new model joint independent subspace analysis jisa we provide full performance analysis of jisa including closed form expressions for minimal mean square error mse fisher information and cramer rao lower bound in the separation of gaussian data the derived mse applies also for non gaussian data when only second order statistics are used we generalize previously known results on iva including its ability to uniquely resolve instantaneous mixtures of real gaussian stationary data and having the same arbitrary permutation at all mixtures numerical experiments validate our theoretical results and show the gain with respect to two competing approaches that either use a finer block partition or a different norm

on the reduction of the multidimensional stationary schrodinger equation to a first order equation and its relation to the pseudoanalytic function theory given a particular solution of a one dimensional stationary schrodinger equation this equation of second order can be reduced to a first order linear ordinary differential equation this is done with the aid of an auxiliary riccati differential equation in the present work we show that the same fact is true in a multidimensional situation also for simplicity we consider the case of two or three independent variables one particular solution of the stationary schrodinger equation allows us to reduce this second order equation to a linear first order quaternionic differential equation as in the one dimensional case this is done with the aid of an auxiliary quaternionic riccati equation the resulting first order quaternionic equation is equivalent to the static maxwell system and is closely related to the dirac equation in the case of two independent variables it is the well known vekua equation from theory of pseudoanalytic or generalized analytic functions nevertheless we show that even in this case it is very useful to consider not only complex valued functions solutions of the vekua equation but complete quaternionic functions in this way the first order quaternionic equation represents two separate vekua equations one of which gives us solutions of the schrodinger equation and the other one can be considered as an auxiliary equation of a simpler structure moreover for the auxiliary equation we always have the corresponding bers generating pair f g the base of the bers theory of pseudoanalytic functions and what is very important the bers derivatives of solutions of the auxiliary equation give us solutions of the main vekua equation and as a consequence of the schrodinger equation based on this fact we obtain an analogue of the cauchy integral theorem for solutions of the stationary schrodinger equation other results from theory of pseudoanalytic functions can be written for solutions of the schrodinger equation moreover for an ample class of potentials in the schrodinger equation which includes for instance all radial potentials this new approach gives us a simple procedure allowing us to obtain an infinite sequence of solutions of the schrodinger equation from one known particular solution

on a factorization of second order elliptic operators and applications we show that given a nonvanishing particular solution of the equation the corresponding differential operator can be factorized into a product of two first order operators the factorization allows us to reduce the above equation to a first order equation which in a two dimensional case is a vekua equation of a special form under quite general conditions on the coefficients p and q we obtain an algorithm which allows us to construct in explicit form positive formal powers solutions of the vekua equation generalizing the usual powers z z0 n n 0 1 this result means that under quite general conditions one can construct an infinite system of exact solutions of the above equation explicitly and moreover at least when p and q are real valued this system will be complete in ker divp grad q in the sense that any solution of the above equation in a simply connected domain can be represented as an infinite series of obtained exact solutions which converges uniformly on any compact subset of finally we give a similar factorization of the operator divp grad q in a multidimensional case and obtain a natural generalization of the vekua equation which is related to second order operators in a similar way as its two dimensional prototype does

causal diagrams for empirical research summary the primary aim of this paper is to show how graphical models can be used as a mathematical language for integrating statistical and subject matter information in particular the paper develops a principled nonparametric framework for causal inference in which diagrams are queried to determine if the assumptions available are sufficient for identifying causal effects from nonexperimental data if so the diagrams can be queried to produce mathematical expressions for causal effects in terms of observed distributions otherwise the diagrams can be queried to suggest additional observations or auxiliary experiments from which the desired inferences can be obtained

stochastic modeling of evaporating sprays within a consistent hybrid joint pdf framework in the present study a framework for modeling two phase evaporating flow is presented which employs an eulerian lagrangian lagrangian approach for the continuous phase a joint velocity composition probability density function pdf method is used opposed to other approaches such pdf methods require no modeling for turbulent convection and chemical source terms for the dispersed phase the pdf of velocity diameter temperature seen gas velocity and seen gas composition is calculated this provides a unified formulation which allows to consistently address the different modeling issues associated with such a system because of the high dimensionality particle methods are employed to solve the pdf transport equations to further enhance computational efficiency a local particle time stepping algorithm is implemented and a particle time averaging technique is employed to reduce statistical and bias errors in comparison to previous studies a significantly smaller number of droplet particles per grid cell can be employed for the computations which rely on two way coupling between the droplet and gas phases the framework was validated using established experimental data and a good overall agreement can be observed

on a transplant operator and explicit construction of cauchy type integral representations for p analytic functions we present a new technique for explicit construction of cauchy kernels and cauchy integral representations for a class of generalized analytic functions and p analytic functions

numerical implementation of mixing and molecular transport in les pdf studies of turbulent reacting flows probability density function pdf methods in combination with large eddy simulations les are a powerful tool for studying turbulent reacting flow problems and we are interested in the implementation of mixing and molecular transport in les pdf methods the numerical methodology used for solution procedure is the hybrid particle mesh method and a fractional step scheme is used to solve for transport reaction and mixing sequentially mixing is modeled using the interaction by exchange with the mean iem model and the effects of molecular transport are incorporated as a mean drift term in the mixing step this methodology avoids spurious production of scalar variance and also allows direct incorporation of differential diffusion effects in this study various numerical implementations of mixing and molecular transport are presented and evaluated using the method of manufactured solutions mms for 1 accuracy 2 detailed conservation 3 realizability and 4 stability moreover the methodology is shown to be successful in capturing the effects of differential diffusion accurately with the additional property of ensuring realizability of species mass fractions finally and most importantly we describe a new variance reduction technique by way of an implicit smoothing methodology this smoothing scheme is shown to satisfy conservation boundedness and regularity criteria moreover for an appropriate choice of the smoothing length scale significant improvements in accuracy can be achieved for an incremental increase in computational cost also it is shown that with smoothing on a length scale greater than the grid size the bias and statistical errors due to there being a finite number of particles in the lagrangian monte carlo simulations scale as n t o t 1 and n t o t 1 2 respectively where n t o t is the total number of particles in the computational domain whereas without smoothing these errors scale as n p c 1 and n p c 1 2 where n p c is the much smaller number of particles in a computational cell

blind separation of independent sources for virtually any source probability density function the blind source separation bss problem consists of the recovery of a set of statistically independent source signals from a set of measurements that are mixtures of the sources when nothing is known about the sources and the mixture structure in the bss scenario of two noiseless real valued instantaneous linear mixtures of two sources an approximate maximum likelihood ml approach has been suggested in the literature which is only valid under certain constraints on the probability density function pdf of the sources in the present paper the expression for this ml estimator is reviewed and generalized to include virtually any source distribution an intuitive geometrical interpretation of the new estimator is also given in terms of the scatter plots of the signals involved an asymptotic performance analysis is then carried out yielding a closed form expression for the estimator asymptotic pdf simulations illustrate the behavior of the suggested estimator and show the accuracy of the asymptotic analysis in addition an extension of the method to the general bss scenario of more than two sources and two sensors is successfully implemented

implicit and explicit schemes for mass consistency preservation in hybrid particle finite volume algorithms for turbulent reactive flows this work addresses the issue of particle mass consistency in large eddy simulation probability density function les pdf methods for turbulent reactive flows numerical schemes for the implicit and explicit enforcement of particle mass consistency pmc are introduced and their performance is examined in a representative les pdf application namely the sandia sydney bluff body flame hm1 a new combination of interpolation schemes for velocity and scalar fields is found to better satisfy pmc than multilinear and fourth order lagrangian interpolation a second order accurate time stepping scheme for stochastic differential equations sde is found to improve pmc relative to euler time stepping which is the first time that a second order scheme is found to be beneficial when compared to a first order scheme in an les pdf application an explicit corrective velocity scheme for pmc enforcement is introduced and its parameters optimized to enforce a specified pmc criterion with minimal corrective velocity magnitudes

a particle formulation for treating differential diffusion in filtered density function methods we present a new approach for treating molecular diffusion in filtered density function fdf methods for modeling turbulent reacting flows the diffusion term accounts for molecular transport in physical space and molecular mixing in composition space conventionally the fdf is represented by an ensemble of particles and transport is modeled by a random walk in physical space there are two significant shortcomings in this transport model 1 the resulting composition variance equation contains a spurious production term and 2 because the random walk is governed by a single diffusion coefficient the formulation cannot account for differential diffusion which can have a first order effect in reacting flows in our new approach transport is simply modeled by a mean drift in the particle composition equation the resulting variance equation contains no spurious production term and differential diffusion is treated easily hence the new formulation reduces to a direct numerical simulation dns in the limit of vanishing filter width a desirable property of any large eddy simulation les approach we use the iem model for mixing it is shown that there is a lower bound on the specified mixing rate such that the boundedness of the compositions is ensured we present a numerical method for solving the particle equations which is second order accurate in space and time obeys detailed conservation enforces the realizability and boundedness constraints and is unconditionally stable

adaptive blind source separation for virtually any source probability density function blind source separation bss aims to recover a set of statistically independent source signals from a set of linear mixtures of the same sources in the noiseless real mixture two source two sensor scenario once the observations are whitened decorrelated and normalized only a givens rotation matrix remains to be identified in order to achieve the source separation in this paper an adaptive estimator of the angle that characterizes such a rotation is derived it is shown to converge to a stable valid separation solution with the only condition that the sum of source kurtosis be distinct from zero an asymptotic performance analysis is carried out resulting in a closed form expression for the asymptotic probability density function of the proposed estimator it is shown how the estimator can be incorporated into a complete adaptive source separation system by combining it with an adaptive prewhitening strategy and how it can be useful in a general bss scenario of more than two signals by means of a pairwise approach a variety of simulations assess the accuracy of the asymptotic results display the properties of the estimator such as its robust fast convergence and compare this on line bss implementation with other adaptive bss procedures

a constrained em algorithm for independent component analysis we introduce a novel way of performing independent component analysis using a constrained version of the expectation maximization em algorithm the source distributions are modeled as d one dimensional mixtures of gaussians the observed data are modeled as linear mixtures of the sources with additive isotropic noise this generative model is fit to the data using constrained em the simpler soft switching approach is introduced which uses only one parameter to decide on the sub or supergaussian nature of the sources we explain how our approach relates to independent factor analysis

hybrid linear and nonlinear complexity pursuit for blind source separation blind source separation bss is an increasingly popular data analysis technique with many applications several methods for bss using the statistical properties of original sources have been proposed for a famous case non gaussianity this leads to independent component analysis ica in this paper we propose a hybrid bss method based on linear and nonlinear complexity pursuit which combines three statistical properties of source signals non gaussianity linear predictability and nonlinear predictability a gradient learning algorithm is presented by minimizing a loss function simulations verify the efficient implementation of the proposed method

preconditioned stochastic gradient descent stochastic gradient descent sgd still is the workhorse for many practical problems however it converges slow and can be difficult to tune it is possible to precondition sgd to accelerate its convergence remarkably but many attempts in this direction either aim at solving specialized problems or result in significantly more complicated methods than sgd this paper proposes a new method to adaptively estimate a preconditioner such that the amplitudes of perturbations of preconditioned stochastic gradient match that of the perturbations of parameters to be optimized in a way comparable to newton method for deterministic optimization unlike the preconditioners based on secant equation fitting as done in deterministic quasi newton methods which assume positive definite hessian and approximate its inverse the new preconditioner works equally well for both convex and nonconvex optimizations with exact or noisy gradients when stochastic gradient is used it can naturally damp the gradient noise to stabilize sgd efficient preconditioner estimation methods are developed and with reasonable simplifications they are applicable to large scale problems experimental results demonstrate that equipped with the new preconditioner without any tuning effort preconditioned sgd can efficiently solve many challenging problems like the training of a deep neural network or a recurrent neural network requiring extremely long term memories

independent vector analysis identification conditions and performance bounds recently an extension of independent component analysis ica from one to multiple datasets termed independent vector analysis iva has been a subject of significant research interest iva has also been shown to be a generalization of hotelling s canonical correlation analysis in this paper we provide the identification conditions for a general iva formulation which accounts for linear nonlinear and sample to sample dependencies the identification conditions are a generalization of previous results for ica and for iva when samples are independently and identically distributed furthermore a principal aim of iva is identification of dependent sources between datasets thus we provide additional conditions for when the arbitrary ordering of the estimated sources can be common across datasets performance bounds in terms of the cramer rao lower bound are also provided for demixing matrices and interference to source ratio the performance of two iva algorithms are compared to the theoretical bounds

an efficient algorithm for scalar pdf modelling in incompressible turbulent flow numerical analysis with evaluation of iem and iecm micro mixing models in this paper we investigate the application of probability density function pdf monte carlo methods to scalar release from small sources in a turbulent flow spanning a large physical domain this is a typical situation encountered when modeling the dispersion of a gaseous substance in the atmosphere monte carlo pdf methods have recently been applied to atmospheric modeling responding to the need for predicting the higher statistics and the concentration pdf generated by the continuous release of reactive and non reactive substances in this work we introduce some optimized numerical techniques based on the paradigm that the main field of interest is the scalar field and not the fluid dynamic field the scalar are considered dynamically passive and the statistical characteristics of the turbulence velocity field are assumed known these techniques are a block structured grid coupled with a particle splitting erasing algorithm and a localized time stepping the proposed technique is different from others presented before since the particle splitting and erasing is done in a more straightforward and consistent manner this method has been applied to the study of scalar dispersion from localized line sources in a canopy generated boundary layer the line source has been treated as a point source in a two dimensional space but the extension to three dimensions is straightforward our framework allows for an evaluation of the effects induced by different levels of discretization in the velocity space of the involved micro mixing model starting from the interaction by the exchange with the mean iem toward the more physically consistent interaction by exchange with the conditional mean iecm therefore aside from the algorithm description and a complete numerical analysis of the code a comparison between the iem and iecm micro mixing models has been investigated

serial updating rule for blind separation derived from the method of scoring in the context of blind source separation the method of scoring based on the inverse of the fisher information matrix fim becomes the serial updating learning rule with an equivariant property this learning rule can be simplified to a low complexity algorithm by using the asymptotic form of the ftm around the equilibrium the simplified learning rule is still general enough to include some existing equivariant blind separation algorithms as its special cases

the parabolic edge reconstruction method perm for lagrangian particle advection we describe a lagrangian particle advection scheme which is intended for use in hybrid finite volume fv large eddy simulation filtered density function les fdf methods for low mach flows but which may also be applicable to unsteady probability density function pdf methods direct numerical simulation dns or any other situation where tracking fluid particles is of concern a key ingredient of the scheme is a subgrid reconstruction of the filtered velocity field with desirable divergence properties which is necessary for accurate evolution of the particle number density we develop reconstructions for 2d and 3d cartesian staggered non uniform grids the reconstructed velocity field is continuous and piecewise parabolic in the velocity component direction in the direction normal to the velocity component the reconstruction is piecewise linear the divergence of the reconstructed field is bilinear in 2d trilinear in 3d within a given cell and consistent with the discrete divergence given by the staggered grid velocities though the reconstructed divergence field may be discontinuous from cell to cell the norm of the differences between the vertex values of the reconstructed divergence for neighboring cells is minimized as a consequence the divergence is everywhere zero for the constant density case a two stage runge kutta scheme is employed for advancement of the particle positions to assess the performance of the scheme we utilize a set of non trivial velocity test functions which are designed to mimic realistic flow fields we show that an advection scheme based on the new velocity reconstruction method is effective at maintaining an accurate particle number density in the particle tracking limit

relative optimization for blind deconvolution we propose a relative optimization framework for quasi maximum likelihood qml blind deconvolution and the relative newton method as its particular instance special hessian structure allows fast newton system construction and solution resulting in a fast convergent algorithm with iteration complexity comparable to that of gradient methods we also propose the use of rational infinite impulse response iir restoration kernels which constitute a richer family of filters than the traditionally used finite impulse response fir kernels we discuss different choices of nonlinear functions that are suitable for deconvolution of super and sub gaussian sources and formulate the conditions under which the qml estimation is stable simulation results demonstrate the efficiency of the proposed methods

blind separation of signals with mixed kurtosis signs using threshold activation functions a parameterized activation function in the form of an adaptive threshold for a single layer neural network which separates a mixture of signals with any distribution except for gaussian is introduced this activation function is particularly simple to implement since it neither uses hyperbolic nor polynomial functions unlike most other nonlinear functions used for blind separation for some specific distributions the stable region of the threshold parameter is derived and optimal values for best separation performance are given if the threshold parameter is made adaptive during the separation process the successful separation of signals whose distribution is unknown is demonstrated and compared against other known methods

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

a theory for learning by weight flow on stiefel grassman manifold recently we introduced the concept of neural network learning on stiefel grassman manifold for multilayer perceptron like networks contributions of other authors have also appeared in the scientific literature about this topic this article presents a general theory for it and illustrates how existing theories may be explained within the general framework proposed here

incorporating diffusion in complex geometries into stochastic chemical kinetics simulations a method is developed for incorporating diffusion of chemicals in complex geometries into stochastic chemical kinetics simulations systems are modeled using the reaction diffusion master equation with jump rates for diffusive motion between mesh cells calculated from the discretization weights of an embedded boundary method since diffusive jumps between cells are treated as first order reactions individual realizations of the stochastic process can be created by the gillespie method numerical convergence results for the underlying embedded boundary method and for the stochastic reaction diffusion method are presented in two dimensions a two dimensional model of transcription translation and nuclear membrane transport in eukaryotic cells is presented to demonstrate the feasibility of the method in studying cell wide biological processes

a generalized contrast function and stability analysis for overdetermined blind separation of instantaneous mixtures in this letter the problem of blind separation of n independent sources from their m linear instantaneous mixtures is considered first a generalized contrast function is defined as a valuable extension of the existing classical and nonsymmetrical contrast functions it is applicable to the overdetermined blind separation m n with an unknown number of sources because not only independent components but also redundant ones are allowed in the outputs of a separation system second a natural gradient learning algorithm developed primarily for the complete case m n is shown to work as well with an n m or m m separating matrix for each optimizes a certain mutual information contrast function finally we present stability analysis for a newly proposed generalized orthogonal natural gradient algorithm which can perform the overdetermined blind separation when n is unknown obtaining an expectable result that its local stability conditions are slightly stricter than those of the conventional natural gradient algorithm using an invertible mixing matrix m n

blind signal separation using steepest descent method a method that significantly improves the convergence rate of the gradient based blind signal separation bss algorithm for convolutive mixtures is proposed the proposed approach is based on the steepest descent algorithm suitable for constrained bss problems where the constraints are included to ease the permutation effects associated with the convolutive mixtures in addition the method is realized using a modified golden search method plus parabolic interpolation and this allows the optimum step size to be determined with only a few calculations of the cost function evaluation of the proposed procedure in simulated environments and in a real room environment shows that the proposed method results in significantly faster convergence for the bss when compared with a fixed step size gradient based algorithm in addition for blind signal extraction where only a main speech source is desired a combined scheme consisting of the proposed bss and a postprocessor such as an adaptive noise canceller offers impressive noise suppression levels while maintaining low target signal distortion levels

independent component analysis over galois fields of prime order we consider the framework of independent component analysis ica for the case where the independent sources and their linear mixtures all reside in a galois field of prime order p similarities and differences from the classical ica framework over the real field are explored we show that a necessary and sufficient identifiability condition is that none of the sources should have a uniform distribution we also show that pairwise independence of the mixtures implies their full mutual independence namely a nonmixing condition in the binary p 2 and ternary p 3 cases but not necessarily in higher order p 3 cases we propose two different iterative separation or identification algorithms one is based on sequential identification of the smallest entropy linear combinations of the mixtures and is shown to be equivariant with respect to the mixing matrix the other is based on sequential minimization of the pairwise mutual information measures we provide some basic performance analysis for the binary p 2 case supplemented by simulation results for higher orders demonstrating advantages and disadvantages of the proposed separation approaches

algorithms for complex ml ica and their stability analysis using wirtinger calculus we derive a class of algorithms for independent component analysis ica based on maximum likelihood ml estimation and perform stability analysis of natural gradient ml ica with and without the constraint for unitary demixing matrix in the process we demonstrate how wirtinger calculus facilitates derivations and most importantly performing second order analysis in the complex domain and eliminates the need for making simplifying assumptions we derive natural gradient complex ml ica update rule and its variant with a unitary constraint as well as a newton algorithm for better convergence behavior the conditions for local stability are derived and studied using a generalized gaussian density ggd source model when the sources are circular and non gaussian we show analytically that both the ml and ml unitary ica update rules converge to the inverse of mixing matrix subject to a phase shift when the sources are noncircular and non gaussian we show that the nonunitary ml ica update rule is more stable than the ml unitary ica update rule when the sources are noncircular gaussians both update rules are stable only when the sources have distinct noncircularity indices simulation results are given to support these results

statistical dynamics of on line independent component analysis the learning dynamics of on line independent component analysis is analysed in the limit of large data dimension we study a simple hebbian learning algorithm that can be used to separate out a small number of non gaussian components from a high dimensional data set the de mixing matrix parameters are confined to a stiefel manifold of tall orthogonal matrices and we introduce a natural gradient variant of the algorithm which is appropriate to learning on this manifold for large input dimension the parameter trajectory of both algorithms passes through a sequence of unstable fixed points each described by a diffusion process in a polynomial potential choosing the learning rate too large increases the escape time from each of these fixed points effectively trapping the learning in a sub optimal state in order to avoid these trapping states a very low learning rate must be chosen during the learning transient resulting in learning time scales of o n2 or o n3 iterations where n is the data dimension escape from each sub optimal state results in a sequence of symmetry breaking events as the algorithm learns each source in turn this is in marked contrast to the learning dynamics displayed by related on line learning algorithms for multilayer neural networks and principal component analysis although the natural gradient variant of the algorithm has nice asymptotic convergence properties it has an equivalent transient dynamics to the standard hebbian algorithm

particle method for turbulent flows integration of stochastic model equations abstract a numerical method is developed to integrate the stochastic differential equations that arise in a particle method for modelling turbulent flows these equations present several challenges the foremost being the presence of multiple time scales the smallest of which can be significantly less than an acceptable time step size t the essence of the approach adopted is to transform and decompose the equations so that the stochastic components which contain the small time scales appear as strictly linear stochastic differential equations analytic solutions to these equations with frozen coefficients are then exploited to produce a stable and accurate scheme when the method is used to advance the properties of n particles the resulting numerical error can be decomposed into three contributions statistical error bias and time stepping error comprehensive tests to study these errors are reported for two test cases a novel variance reduction technique is described that significantly reduces the statistical error which scales as n 1 2 in general the bias is smaller and scales as n 1 in accord with a simple analysis the time stepping error is less than 1 for a non dimensional time step of 1 32 which may be several times larger than the smallest time scale over the range of time step size investigated the dominant time stepping error varies as t 3 2 the method has the requisite stability accuracy and efficiency for incorporation in multi dimensional particle methods

algorithms for nonnegative independent component analysis we consider the task of solving the independent component analysis ica problem x as given observations x with a constraint of nonnegativity of the source random vector s we refer to this as nonnegative independent component analysis and we consider methods for solving this task for independent sources with nonzero probability density function pdf p s down to s 0 it is sufficient to find the orthonormal rotation y wz of prewhitened sources z vx which minimizes the mean squared error of the reconstruction of z from the rectified version y sup of y we suggest some algorithms which perform this both based on a nonlinear principal component analysis pca approach and on a geodesic search method driven by differential geometry considerations we demonstrate the operation of these algorithms on an image separation problem which shows in particular the fast convergence of the rotation and geodesic methods and apply the approach to a musical audio analysis task

globally convergent blind source separation based on a multiuser kurtosis maximization criterion we consider the problem of recovering blindly i e without the use of training sequences a number of independent and identically distributed source user signals that are transmitted simultaneously through a linear instantaneous mixing channel the received signals are hence corrupted by interuser interference iui and we can model them as the outputs of a linear multiple input multiple output mimo memoryless system assuming the transmitted signals to be mutually independent i i d and to share the same non gaussian distribution a set of necessary and sufficient conditions for the perfect blind recovery up to scalar phase ambiguities of all the signals exists and involves the kurtosis as well as the covariance of the output signals we focus on a straightforward blind constrained criterion stemming from these conditions from this criterion we derive an adaptive algorithm for blind source separation which we call the multiuser kurtosis muk algorithm at each iteration the algorithm combines a stochastic gradient update and a gram schmidt orthogonalization procedure in order to satisfy the criterion s whiteness constraints a performance analysis of its stationary points reveals that the muk algorithm is free of any stable undesired local stationary points for any number of sources hence it is globally convergent to a setting that recovers them all

waveform preserving blind estimation of multiple independent sources the problem of blind estimation of source signals is to estimate the source signals without knowing the characteristics of the transmission channel it is shown that the minimum variance unbiased estimates can be obtained if and only if the transmission channel can be identified blindly it is shown that the channel can be blindly identified if and only if there is not more than one gaussian source this condition suggests that waveform preserving blind estimation can be achieved over a wide range of signal processing applications including those cases in which the source signals have identical nongaussian distributions the constructive proof of the necessary and sufficient condition serves as a foundation for the development of waveform preserving blind signal estimation algorithms examples are presented to demonstrate the applications of the theoretical results

directlingam a direct method for learning a linear non gaussian structural equation model structural equation models and bayesian networks have been widely used to analyze causal relations between continuous variables in such frameworks linear acyclic models are typically used to model the data generating process of variables recently it was shown that use of non gaussianity identifies the full structure of a linear acyclic model that is a causal ordering of variables and their connection strengths without using any prior knowledge on the network structure which is not the case with conventional methods however existing estimation methods are based on iterative search algorithms and may not converge to a correct solution in a finite number of steps in this paper we propose a new direct method to estimate a causal ordering and connection strengths based on non gaussianity in contrast to the previous methods our algorithm requires no algorithmic parameters and is guaranteed to converge to the right solution within a small fixed number of steps if the data strictly follows the model that is if all the model assumptions are met and the sample size is infinite

a hybrid algorithm for the joint pdf equation of turbulent reactive flows in this paper a new particle finite volume hybrid algorithm for the joint velocityfrequency composition pdf method for turbulent reactive flows is presented this method is a combination of a finite volume scheme and a particle method the finitevolume scheme is used to solve the reynolds averaged navier stokes equations and the particle method to solve the joint pdf transport equation the motivation is to reduce the bias and the statistical error and to have an algorithm which is more efficient than stand alone particle mesh methods therefore in the particle method we use the smoother mean densityh i and favre averaged velocity u fields computed

estimating functions of independent component analysis for temporally correlated signals this article studies a general theory of estimating functions of independent component analysis when the independent source signals are temporarily correlated estimating functions are used for deriving both batch and on line learning algorithms and they are applicable to blind cases where spatial and temporal probability structures of the sources are unknown most algorithms proposed so far can be analyzed in the framework of estimating functions an admissible class of estimating functions is derived and related efficient on line learning algorithms are introduced we analyze dynamical stability and statistical efficiency of these algorithms different from the independently and identically distributed case the algorithms work even when only the second order moments are used the method of simultaneous diagonalization of cross covariance matrices is also studied from the point of view of estimating functions

performance analysis of the fastica algorithm and crame spl acute r rao bounds for linear independent component analysis the fastica or fixed point algorithm is one of the most successful algorithms for linear independent component analysis ica in terms of accuracy and computational complexity two versions of the algorithm are available in literature and software a one unit deflation algorithm and a symmetric algorithm the main result of this paper are analytic closed form expressions that characterize the separating ability of both versions of the algorithm in a local sense assuming a good initialization of the algorithms and long data records based on the analysis it is possible to combine the advantages of the symmetric and one unit version algorithms and predict their performance to validate the analysis a simple check of saddle points of the cost function is proposed that allows to find a global minimum of the cost function in almost 100 simulation runs second the crame spl acute r rao lower bound for linear ica is derived as an algorithm independent limit of the achievable separation quality the fastica algorithm is shown to approach this limit in certain scenarios extensive computer simulations supporting the theoretical findings are included

kernel independent component analysis we present a class of algorithms for independent component analysis ica which use contrast functions based on canonical correlations in a reproducing kernel hilbert space on the one hand we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence on the other hand building on recent developments in kernel methods we show that these criteria and their derivatives can be computed efficiently minimizing these criteria leads to flexible and robust algorithms for ica we illustrate with simulations involving a wide variety of source distributions showing that our algorithms outperform many of the presently known algorithms

fast and robust fixed point algorithms for independent component analysis independent component analysis ica is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible we use a combination of two different approaches for linear ica comon s information theoretic approach and the projection pursuit approach using maximum entropy approximations of differential entropy we introduce a family of new contrast functions for ica these contrast functions enable both the estimation of the whole decomposition by minimizing mutual information and estimation of individual independent components as projection pursuit directions the statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model and it is shown how to choose contrast functions that are robust and or of minimum variance finally we introduce simple fixed point algorithms for practical optimization of the contrast functions

penalty function based joint diagonalization approach for convolutive blind separation of nonstationary sources a new approach for convolutive blind source separation bss by explicitly exploiting the second order nonstationarity of signals and operating in the frequency domain is proposed the algorithm accommodates a penalty function within the cross power spectrum based cost function and thereby converts the separation problem into a joint diagonalization problem with unconstrained optimization this leads to a new member of the family of joint diagonalization criteria and a modification of the search direction of the gradient based descent algorithm using this approach not only can the degenerate solution induced by a null unmixing matrix and the effect of large errors within the elements of covariance matrices at low frequency bins be automatically removed but in addition a unifying view to joint diagonalization with unitary or nonunitary constraint is provided numerical experiments are presented to verify the performance of the new method which show that a suitable penalty function may lead the algorithm to a faster convergence and a better performance for the separation of convolved speech signals in particular in terms of shape preservation and amplitude ambiguity reduction as compared with the conventional second order based algorithms for convolutive mixtures that exploit signal nonstationarity

adaptive modal identification of structures with equivariant adaptive separation via independence approach abstract an efficient output only blind source separation bss method was recently introduced for the modal identification of structures bss procedures recover a set of independent sources from their unknown linear mixtures when only mixtures are observed batch data is required for the separation in traditional blind source separation methods these algorithms are however unfavorable as some sets of data are observed one after another in this paper an adaptive blind source separation technique equivariant adaptive separation via independence easi is introduced to overcome the mentioned disadvantage within the structures the easi algorithm is beneficial as it can provide solutions to real time problems while also update the un mixing matrix for each step easi not only avoids increases in size of the relevant matrices and vectors but also decreases the analysis time a synthetic example and a benchmark structure have been used in this paper to better investigate the efficiency of the proposed method the simulation results demonstrate the effectiveness of the easi algorithm in on line identification of modal parameters of structures

causality in linear nongaussian acyclic models in the presence of latent gaussian confounders lingam has been successfully applied to some real world causal discovery problems nevertheless causal sufficiency is assumed that is there is no latent confounder of the observations which may be unrealistic for real world problems taking into the consideration latent confounders will improve the reliability and accuracy of estimations of the real causal structures in this letter we investigate a model called linear nongaussian acyclic models in the presence of latent gaussian confounders lingam gc which can be seen as a specific case of lvlingam this model includes the latent confounders which are assumed to be independent gaussian distributed and statistically independent of the disturbances to tackle the causal discovery problem of this model first we propose a pairwise cumulant based measure of causal directions for cause effect pairs we prove that in spite of the presence of latent gaussian confounders the causal direction of the observed cause effect pair can be identified under the mild condition that the disturbances are simultaneously supergaussian or subgaussian we propose a simple and efficient method to detect the violation of this condition we extend our work to multivariate causal network discovery problems specifically we propose algorithms to estimate the causal network structure including causal ordering and causal strengths using an iterative root finding removing scheme based on pairwise measure to address the redundant edge problem due to the finite sample size effect we develop an efficient bootstrapping based pruning algorithm experiments on synthetic data and real world data have been conducted to show the applicability of our model and the effectiveness of our proposed algorithms

an expectation maximization method for spatio temporal blind source separation using an ar mog source model in this paper we develop a maximum likelihood ml spatio temporal blind source separation bss algorithm where the temporal dependencies are explained by assuming that each source is an autoregressive ar process and the distribution of the associated independent identically distributed i i d innovations process is described using a mixture of gaussians unlike most ml methods the proposed algorithm takes into account both spatial and temporal information optimization is performed using the expectation maximization em method the source model is adapted to maximize the likelihood and the update equations have a simple analytical form the proposed method which we refer to as autoregressive mixture of gaussians ar mog outperforms nine other methods for artificial mixtures of real audio we also show results for using ar mog to extract the fetal cardiac signal from real magnetocardiographic mcg data

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

a linear non gaussian acyclic model for causal discovery in recent years several methods have been proposed for the discovery of causal structure from non experimental data such methods make various assumptions on the data generating process to facilitate its identification from purely observational data continuing this line of research we show how to discover the complete causal structure of continuous valued data under the assumptions that a the data generating process is linear b there are no unobserved confounders and c disturbance variables have non gaussian distributions of non zero variances the solution relies on the use of the statistical method known as independent component analysis and does not require any pre specified time ordering of the variables we provide a complete matlab package for performing this lingam analysis short for linear non gaussian acyclic model and demonstrate the effectiveness of the method using artificially generated data and real world data

a general mass consistency algorithm for hybrid particle finite volume pdf methods an algorithm is devised to maintain a correct spatial distribution of computational particles in hybrid particle finite volume fv probability density function pdf methods for chemically reacting turbulent flows the approach is by design compatible with existing fv computational fluid dynamics cfd codes that are used to model practical engineering flows in complex geometric configurations the algorithm is suitable for general three dimensional incompressible or compressible steady or time dependent flows using structured or unstructured stationary or deforming computational meshes it is compatible with a variety of element shapes commonly used in research and engineering cfd codes including hexahedra prisms and tetrahedra robustness accuracy and efficiency of the approach are demonstrated via computations for several two and three dimensional steady and unsteady flow configurations using computational meshes that vary in element type and in mesh quality both composition pdf and velocity pdf methods are employed this work broadens the accessibility of pdf methods for practical turbulent combustion systems

a multiblock joint pdf finite volume hybrid algorithm for the computation of turbulent flows in complex geometries a new hybrid joint probability density function jpdf solution algorithm for turbulent flows in complex 3d geometries is presented the main focus is to demonstrate the applicability of jpdf methods for complex flows as observed in industrial applications all elements of the algorithm are explained in detail and extensive validation studies are presented a multiblock finite volume solver capable of handling globally unstructured locally structured grids was implemented together with a lagrangian particle method efficient and robust particle management and accurate schemes for estimation and interpolation of particle statistics have been developed for numerical efficiency particle sub time stepping and an implicit finite volume solver are applied a fast coupling strategy was developed together with a multigrid method that allows very fast convergence on refined grids comparison with an established jpdf code for a bluff body stabilized flow shows very good agreement furthermore robustness and consistency of the algorithm for turbulent flow simulations with complex geometry is demonstrated

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

sequential blind source separation based exclusively on second order statistics developed for a class of periodic signals a sequential algorithm for the blind separation of a class of periodic source signals is introduced in this paper the algorithm is based only on second order statistical information and exploits the assumption that the source signals have distinct periods separation is performed by sequentially converging to a solution which in effect diagonalizes the output covariance matrix constructed at a lag corresponding to the fundamental period of the source we select the one with the smallest period simulation results for synthetic signals and real electrocardiogram recordings show that the proposed algorithm has the ability to restore statistical independence and its performance is comparable to that of the equivariant adaptive source separation easi algorithm a benchmark high order statistics based sequential algorithm with similar computational complexity the proposed algorithm is also shown to mitigate the limitation that the easi algorithm can separate at most one gaussian distributed source furthermore the steady state performance of the proposed algorithm is compared with that of easi and the block based second order blind identification sobi method

markovian source separation a maximum likelihood ml approach is used to separate the instantaneous mixtures of temporally correlated independent sources with neither preliminary transformation nor a priori assumption about the probability distribution of the sources a markov model is used to represent the joint probability density of successive samples of each source the joint probability density functions are estimated from the observations using a kernel method for the special case of autoregressive models the theoretical performance of the algorithm is computed and compared with the performance of second order algorithms and i i d based separation algorithms

assessment of numerical accuracy of pdf monte carlo methods for turbulent reacting flows this study is to explore the numerical features of a particle mesh algorithm developed for a stand alone joint velocity frequency composition pdf method for turbulent reactive flows numerical experiments are performed on a piloted jet nonpremixed turbulent flame of methane to characterize and quantify various numerical errors in terms of numerical parameters number of particles per cellnpc number of cells m2 and time step t first a stationary solution is obtained and is verified to be independent of the time step t then the total numerical error is identified as statistical error bias and discretization error it is revealed that the statistical error converges as npc 1 2 and the bias as npc 1 the statistical error can be reduced by time averaging or by performing multiple independent simulation e g with a parallelized program finally the scheme is shown to be second order accurate the spatial discretization error converging as m 2 a modified turbulence frequency model based on the turbulence production to dissipation ratio is shown to improve the numerical behavior of the turbulence model these results demonstrate that the particle mesh method is convergent also the optimal numerical parameters minimizing computational cost subject to a specified error tolerance are estimated an error reduction scheme similar to richardson extrapolation is proposed and shown to be quite effective in reducing the deterministic error

independent factor analysis we introduce the independent factor analysis ifa method for recovering independent hidden sources from their observed mixtures ifa generalizes and unifies ordinary factor analysis fa principal component analysis pca and independent component analysis ica and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy ifa is a two step procedure in the first step the source densities mixing matrix and noise covariance are estimated from the observed data by maximum likelihood for this purpose we present an expectation maximization em algorithm which performs unsupervised learning of an associated probabilistic model of the mixing situation each source in our model is described by a mixture of gaussians thus all the probabilistic calculations can be performed analytically in the second step the sources are reconstructed from the observed data by an optimal nonlinear estimator a variational approximation of this algorithm is derived for cases with a large number of sources where the exact algorithm becomes intractable our ifa algorithm reduces to the one for ordinary fa when the sources become gaussian and to an em algorithm for pca in the zero noise limit we derive an additional em algorithm specifically for noiseless ifa this algorithm is shown to be superior to ica since it can learn arbitrary source densities from the data beyond blind separation ifa can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding

new applications of pseudoanalytic function theory to the dirac equation in the present work we establish a simple relation between the dirac equation with a scalar and an electromagnetic potential in a two dimensional case and a pair of decoupled vekua equations in general these vekua equations are bicomplex however we show that the whole theory of pseudoanalytic functions without modifications can be applied to these equations under a certain nonrestrictive condition as an example we formulate the similarity principle which is the central reason why a pseudoanalytic function and as a consequence a spinor field depending on two space variables share many of the properties of analytic functions one of the surprising consequences of the established relation with pseudoanalytic functions consists in the following result consider the dirac equation with a scalar potential depending on one variable with fixed energy and mass in general this equation cannot be solved explicitly even if one looks for wavefunctions of one variable nevertheless for such dirac equation we obtain an algorithmically simple procedure for constructing in explicit form a complete system of exact solutions depending on two variables these solutions generalize the system of powers 1 z z2 in complex analysis and are called formal powers with their aid any regular solution of the dirac equation can be represented by its taylor series in formal powers

new criteria for blind deconvolution of nonminimum phase systems channels a necessary and sufficient condition for blind deconvolution without observing the input of nonminimum phase linear time invariant systems channels is derived based on this condition several optimization criteria are proposed and their solution is shown to correspond to the desired response these criteria involve the computation only of second and fourth order moments implying a simple tap update procedure the proposed methods are universal in the sense that they do not impose any restrictions on the probability distribution of the unobserved input sequence it is shown that in several important cases e g when the additive noise is gaussian the proposed criteria are essentially unaffected

faster independent component analysis by preconditioning with hessian approximations independent component analysis ica is a technique for unsupervised exploration of multichannel data that is widely used in observational sciences in its classic form ica relies on modeling the data as linear mixtures of nongaussian independent sources the maximization of the corresponding likelihood is a challenging problem if it has to be completed quickly and accurately on large sets of real data this problem has been addressed by resorting to quasi newton methods which rely on sparse approximations of the hessian of the log likelihood however those approximations are not accurate when the ica model does not hold exactly as is often the case for real datasets we propose a new algorithm dubbed picard which makes use of sparse approximate hessians only as a preconditioner to the l bfgs algorithm refining the hessian approximation from a memory of the past iterates extensive numerical comparisons to several algorithms of the same class demonstrate the superior performance of the proposed technique especially on real data

stochastic trapping in a solvable model of on line independent component analysis previous analytical studies of on line independent component analysis ica learning rules have focused on asymptotic stability and efficiency in practice the transient stages of learning are often more significant in determining the success of an algorithm this is demonstrated here with an analysis of a hebbian ica algorithm which can find a small number of nongaussian components given data composed of a linear mixture of independent source signals an idealized data model is considered in which the sources comprise a number of nongaussian and gaussian sources and a solution to the dynamics is obtained in the limit where the number of gaussian sources is infinite previous stability results are confirmed by expanding around optimal fixed points where a closed form solution to the learning dynamics is obtained however stochastic effects are shown to stabilize otherwise unstable suboptimal fixed points conditions required to destabilize one such fixed point are obtained for the case of a single nongaussian component indicating that the initial learning rate required to escape successfully is very low o n 2 where n is the data dimension resulting in very slow learning typically requiring o n3 iterations simulations confirm that this picture holds for a finite system

statistical inference for independent component analysis application to structural var models abstract the well known problem of non identifiability of structural var models disappears if the structural shocks are independent and if at most one of them is gaussian in that case the relevant estimation technique is the independent component analysis ica since the introduction of ica by comon 1994 various semi parametric estimation methods have been proposed for orthogonalizing the error terms these methods include pseudo maximum likelihood pml approaches and recursive pml the aim of our paper is to derive the asymptotic properties of the pml approaches in particular to study their consistency we conduct monte carlo studies exploring the relative performances of these methods finally an application based on real data shows that structural var models can be estimated without additional identification restrictions in the non gaussian case and that the usual restrictions can be tested

parcelingam a causal ordering method robust against latent confounders we consider learning a causal ordering of variables in a linear nongaussian acyclic model called lingam several methods have been shown to consistently estimate a causal ordering assuming that all the model assumptions are correct but the estimation results could be distorted if some assumptions are violated in this letter we propose a new algorithm for learning causal orders that is robust against one typical violation of the model assumptions latent confounders the key idea is to detect latent confounders by testing independence between estimated external influences and find subsets parcels that include variables unaffected by latent confounders we demonstrate the effectiveness of our method using artificial data and simulated brain imaging data

a hybridized discontinuous galerkin framework for high order particle mesh operator splitting of the incompressible navier stokes equations abstract a generic particle mesh method using a hybridized discontinuous galerkin hdg framework is presented and validated for the solution of the incompressible navier stokes equations building upon particle in cell concepts the method is formulated in terms of an operator splitting technique in which lagrangian particles are used to discretize an advection operator and an eulerian mesh based hdg method is employed for the constitutive modeling to account for the inter particle interactions key to the method is the variational framework provided by the hdg method this allows to formulate the projections between the lagrangian particle space and the eulerian finite element space in terms of local i e cellwise l 2 projections efficiently furthermore exploiting the hdg framework for solving the constitutive equations results in velocity fields which excellently approach the incompressibility constraint in a local sense by advecting the particles through these velocity fields the particle distribution remains uniform over time obviating the need for additional quality control the presented methodology allows for a straightforward extension to arbitrary order spatial accuracy on general meshes a range of numerical examples shows that optimal convergence rates are obtained in space and given the particular time stepping strategy second order accuracy is obtained in time the model capabilities are further demonstrated by presenting results for the flow over a backward facing step and for the flow around a cylinder

regular article a consistent hybrid finite volume particle method for the pdf equations of turbulent reactive flows the paper describes a new hybrid finite volume fv particle method developed for the solution of the pdf equations for statistically stationary turbulent reactive flows in this approach the conservation equations for mean mass momentum and energy conservation are solved by a fv method while a particle algorithm is employed to solve the fluctuating velocity turbulence frequency compositions joint pdf transport equation the mean velocity and pressure are supplied to the particle code by the fv code which in turn obtains all the reynolds stresses the scalar fluxes and the reaction terms needed in the fv code an important feature of the method is the complete consistency between the set of equations solved by the fv and particle methods the algorithmic and numerical issues arising in the development of the hybrid method are studied in the simple setting of the stochastic ideal flow equations numerical results are obtained for 1d reactive stochastic ideal flow to demonstrate numerical properties of the method the total numerical error is identified as statistical error bias spatial truncation error and temporal truncation error in contrast to the self contained particle method the bias is found to be negligibly small it is shown that all the numerical errors converge at the expected rates finally the global convergence of the hybrid method is demonstrated and the optimal strategy for time averaging that gives the best global convergence rate is investigated