statistical mechanics and phase transitions in clustering a new approach to clustering based on statistical physics is presented the problem is formulated as fuzzy clustering and the association probability distribution is obtained by maximizing the entropy at a given average variance the corresponding lagrange multiplier is related to the temperature and motivates a deterministic annealing process where the free energy is minimized at each temperature critical temperatures are derived for phase transitions when existing clusters split it is a hierarchical clustering estimating the most probable cluster parameters at various average variances

least squares quadratic b spline finite element method for the regularised long wave equation abstract an approximate solution consisting of a combination of the quadratic b spline functions is incorporated into the least squares method an application of this method is presented for computing the solution of the regularised long wave rlw equation quadratic b spline solution of the rlw equation leads to tridiagonal matrix system which is solved easily by using the thomas algorithm performance of this scheme is tested by obtaining a solitary wave solution of the equation and by studying the development of the undular bore numerical results of the proposed algorithm is shown to have higher accuracy in terms of l 2 error norm compared studying the migration of the solitary wave a fourier stability analysis of the method is investigated

notes on bell sejnowski pdf matching neuron this article investigates the behavior of a single input single unit neuron model of the bell sejnowski class which learn through the maximum entropy principle in order to understand its probability density function matching ability

application of galerkin s method to equal width wave equation the non linear equal width equation is solved by galerkin s method using linear finite elements a linear recurrence relationship for the numerical solution of the resulting system of ordinary differential equations is found employing the crank nicolson approach including a product approximation the three invariants of the motion are calculated to determine the conservation properties of the system l 2 and l error norms are used to measure differences between the exact and numerical solutions the simulations of solitary wave motion are used to determine the properties of the algorithm finally the development of an undular bore is studied

ica using spacings estimates of entropy this paper presents a new algorithm for the independent components analysis ica problem based on an efficient entropy estimator like many previous methods this algorithm directly minimizes the measure of departure from independence according to the estimated kullback leibler divergence between the joint distribution and the product of the marginal distributions we pair this approach with efficient entropy estimators from the statistics literature in particular the entropy estimator we use is consistent and exhibits rapid convergence the algorithm based on this estimator is simple computationally efficient intuitively appealing and outperforms other well known algorithms in addition the estimator s relative insensitivity to outliers translates into superior performance by our ica algorithm on outlier tests we present favorable comparisons to the kernel ica fast ica jade and extended infomax algorithms in extensive simulations we also provide public domain source code for our algorithms

code vector density in topographic mappings scalar case the author derives some new results that build on his earlier work 1989 of combining vector quantization vq theory and topographic mapping tm theory a vq model with a noisy transmission medium is used to model the processes that occur in tms which leads to the standard tm training algorithm albeit with a slight modification to the encoding process to emphasize this difference the model is called a topographic vector quantizer tvq in the continuum limit of the one dimensional scalar tvq it is found that the density of code vectors is proportional to p x sup a alpha 1 3 assuming that the transmission medium introduces additive noise with a zero mean symmetric monotically decreasing probability density this result is dramatically different from the result that is predicted when the standard tm training algorithm is used with a uniform symmetric neighborhood n n and it is noted that this difference arises entirely from using minimum distortion rather than nearest neighbor encoding

probabilistic self organizing maps for continuous data the original self organizing feature map did not define any probability distribution on the input space however the advantages of introducing probabilistic methodologies into self organizing map models were soon evident this has led to a wide range of proposals which reflect the current emergence of probabilistic approaches to computational intelligence the underlying estimation theories behind them derive from two main lines of thought the expectation maximization methodology and stochastic approximation methods here we present a comprehensive view of the state of the art with a unifying perspective of the involved theoretical frameworks in particular we examine the most commonly used continuous probability distributions self organization mechanisms and learning schemes special emphasis is given to the connections among them and their relative advantages depending on the characteristics of the problem at hand furthermore we evaluate their performance in two typical applications of self organizing maps classification and visualization

solitary waves of the regularized long wave equation abstract a finite element solution of the regularised long wave equation based on galerkin s method using cubic splines as element shape functions is set up a linear stability analysis shows the scheme to be unconditionally stable test problems including the migration and interaction of solitary waves are used to validate the method which is found to be accurate and efficient the three invariants of the motion are evaluated to determine the conservation properties of the algorithm the temporal evolution of a maxwellian initial pulse is then studied

petrov galerkin methods for nonlinear dispersive waves abstract petrov galerkin methods based on piecewise linear interpolants for the korteweg de vries and related equations are studied both accuracy and stability are analysed for the linearised case it is shown that in the nonlinear case the order of accuracy of the standard galerkin procedure is reduced and an alternative technique is therefore proposed which retains the fourth order accuracy in space this is found to perform well when compared with finite difference or other finite element schemes

entropy based kernel mixture modeling for topographic map formation a new information theoretic learning algorithm for kernel based topographic map formation is introduced in the one dimensional case the algorithm is aimed at uniformizing the cumulative distribution of the kernel mixture densities by maximizing its differential entropy a nonparametric differential entropy estimator is used on which normalized gradient ascent is performed both differentiable and nondifferentiable kernels are in principle supported such as gaussian and rectangular on off kernels the relation is shown with joint entropy maximization of the kernel outputs the learning algorithm s performance is assessed and compared with the theoretically optimal performance a fixed point rule is derived for the case of heterogeneous kernel mixtures finally an extension of the algorithm to the multidimensional case is suggested

best asymptotic normality of the kernel density entropy estimator for smooth densities in the random sampling setting we estimate the entropy of a probability density distribution by the entropy of a kernel density estimator using the double exponential kernel under mild smoothness and moment conditions we show that the entropy of the kernel density estimator equals a sum of independent and identically distributed i i d random variables plus a perturbation which is asymptotically negligible compared to the parametric rate n sup 1 2 an essential part in the proof is obtained by exhibiting almost sure bounds for the kullback leibler divergence between the kernel density estimator and its expected value the basic technical tools are doob s submartingale inequality and convexity jensen s inequality

how to generate ordered maps by maximizing the mutual information between input and output signals a learning rule that performs gradient ascent in the average mutual information between input and an output signal is derived for a system having feedforward and lateral interactions several processes emerge as components of this learning rule hebb like modification and cooperation and competition among processing nodes topographic map formation is demonstrated using the learning rule an analytic expression relating the average mutual information to the response properties of nodes and their geometric arrangement is derived in certain cases this yields a relation between the local map magnification factor and the probability distribution in the input space the results provide new links between unsupervised learning and information theoretic optimization in a system whose properties are biologically motivated

a space time least square finite element scheme for advection diffusion equations abstract a space time least square finite element scheme is presented for the advection diffusion problems at moderate to high peclet numbers this scheme is designed to eliminate spurious oscillations and can be used to define the steady state solution as the asymptotic transient solution for large time numerical results using linear elements in a 1d space and bilinear elements in a 2d space demonstrate the accuracy and the stability of the new scheme

self organizing mixture networks for probability density estimation a self organizing mixture network somn is derived for learning arbitrary density functions the network minimizes the kullback leibler information metric by means of stochastic approximation methods the density functions are modeled as mixtures of parametric distributions a mixture needs not to be homogenous i e it can have different density profiles the first layer of the network is similar to kohonen s self organizing map som but with the parameters of the component densities as the learning weights the winning mechanism is based on maximum posterior probability and updating of the weights is limited to a small neighborhood around the winner the second layer accumulates the responses of these local nodes weighted by the learned mixing parameters the network possesses a simple structure and computational form yet yields fast and robust convergence the network has a generalization ability due to the relative entropy criterion used applications to density profile estimation and pattern classification are presented the somn can also provide an insight to the role of neighborhood function used in the som

least squares finite element method for the advection diffusion equation the space time least squares finite element methods are constructed for the advection diffusion equation by using both linear shape functions and quadratic b spline shape functions two test problems are studied to demonstrate the accuracy of the present methods results of the two schemes have been compared

explicit formula for the limit of a difference approximation we consider the conservation law u t log ae u be u x 0 where a 0 b 0 a b 1 with initial condition u x 0 u o x in the quarter plane x 0 t 0 we are interested in studying the limit of the approximate solution defined by lax s difference scheme and we discuss the problem of percribing u 0 t t arbitrarily

sums of functions of nearest neighbor distances moment bounds limit theorems and a goodness of fit test abstract the limiting behavior of sums of functions of nearest neighbor distances is studied for an m dimensional sample a central limit theorem and moment bounds for such sums and an invariance principle for the empirical process of nearest neighbor distances are both established as a consequence the asymptotic behavior of a practicable goodness of fit test is obtained based on nearest neighbor distances

the burgers equation on the semiline with general boundary conditions at the origin a technique is given to solve the initial boundary value problem for the burgers equation ut x t uxx x t 2 ux x t u x t on the semiline 0 x with the general boundary condition at the origin h u 0 t ux 0 t t 0 here to solve means to reduce to an equation in one variable only this equation is generally nonlinear and integrodifferent ial it comes in several equivalent avatars which contain nontrivially a free parameter whose value can be assigned arbitrarily since the solution of the equation is independent of it in the special case when h y z t a t y b t z y2 f t which is the case relevant for most applications the equations reduce to linear integral equations of volterra type which can in fact be solved by quadratures if a t f t c1 and b t f t c2 are time independent

joint entropy maximization in kernel based topographic maps a new learning algorithm for kernel based topographic map formation is introduced the kernel parameters are adjusted individually so as to maximize the joint entropy of the kernel outputs this is done by maximizing the differential entropies of the individual kernel outputs given that the map s output redundancy due to the kernel overlap needs to be minimized the latter is achieved by minimizing the mutual information between the kernel outputs as a kernel the radial incomplete gamma distribution is taken since for a gaussian input density the differential entropy of the kernel output will be maximal since the theoretically optimal joint entropy performance can be derived for the case of nonoverlapping gaussian mixture densities a new clustering algorithm is suggested that uses this optimum as its null distribution finally it is shown that the learning algorithm is similar to one that performs stochastic gradient descent on the kullback leibler divergence for a heteroskedastic gaussian mixture density model

burger s equation in the quarter plane a formula for the weak limit on etudie la limite pour e 0 des solutions du probleme aux valeurs limites et initiales u t e ue 2 2 x eu xx e 2 ue x 0 u 0 x ue 0 t u b t dans d x t x 0 t 0 on suppose que u 0 x l 1 0 et u b c 0

local discontinuous galerkin methods for nonlinear schrodinger equations in this paper we develop a local discontinuous galerkin method to solve the generalized nonlinear schrodinger equation and the coupled nonlinear schrodinger equation l 2 stability of the schemes are obtained for both of these nonlinear equations numerical examples are shown to demonstrate the accuracy and capability of these methods

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

cooperative information maximization with gaussian activation functions for self organizing maps in this paper we propose a new information theoretic method to produce explicit self organizing maps soms competition is realized by maximizing mutual information between input patterns and competitive units competitive unit outputs are computed by the gaussian function of distance between input patterns and competitive units a property of this gaussian function is that as distance becomes smaller a neuron tends to fire strongly cooperation processes are realized by taking into account the firing rates of neighboring neurons we applied our method to uniform distribution learning chemical compound classification and road classification experimental results confirmed that cooperation processes could significantly increase information content in input patterns when cooperative operations are not effective in increasing information mutual information as well as entropy maximization is used to increase information experimental results showed that entropy maximization could be used to increase information and to obtain clearer soms because competitive units are forced to be equally used on average