evaluating kohonen s learning rule an approach through genetic algorithms this paper examines the technical foundations of the self organising map som it compares kohonen s heuristic based training algorithm with direct optimisation of a locally weighted distortion index also used by kohonen direct optimisation is achieved through a genetic algorithm ga although gas have been used before with the som this has not been done in conjunction with the distortion index comparing heuristic based training and direct optimisation for the som is analogous to comparing the backpropagation algorithm for feedforward networks with direct optimisation of rms error our experiments reveal lower values of the distortion index with direct optimisation as to whether the heuristic based algorithm is able to provide an approximation to gradient descent our results suggest the answer should be in the negative theorems for one dimensional and for square maps indicate that different point densities will emerge for the two training approaches our findings are in accordance with these results

symmetrically trimmed least squares estimation for tobit models this paper proposes alternatives to maximum likelihood estimation of the censored and truncated regression models known to economists as tobit models the proposed estimators are based upon symmetric censoring or truncation of the upper tail of the distribution of the dependent variable unlike methods based on the assumption of identically distributed gaussian errors the estimators are semiparametric in the sense that they are consistent and asymptotically normal for a wide class of symmetric error distributions with heteroskedasticity of unknown form the paper gives the regularity conditions and proofs of these large sample properties demonstrates how to construct consistent estimators of the asymptotic covariance matrices and presents the results of a simulation study for the censored case extensions and limitations of the approach are also considered

on using prototype reduction schemes to optimize kernel based nonlinear subspace methods abstract the subspace method of pattern recognition is a classification technique in which pattern classes are specified in terms of linear subspaces spanned by their respective class based basis vectors to overcome the limitations of the linear methods kernel based nonlinear subspace kns methods have been recently proposed in the literature in kns the kernel principal component analysis kpca has been employed to get principal components not in an input space but in a high dimensional space where the components of the space are nonlinearly related to the input variables the length of projections onto the basis vectors in the kpca are computed using a kernel matrix k whose dimension is equivalent to the number of sample data points clearly this is problematic especially for large data sets in this paper we suggest a computationally superior mechanism to solve the problem rather than define the matrix k with the whole data set and compute the principal components we propose that the data be reduced into a smaller representative subset using a prototype reduction scheme prs since a prs has the capability of extracting vectors that satisfactorily represent the global distribution structure we demonstrate that data points which are ineffective in the classification can be eliminated to obtain a reduced kernel matrix k without degrading the performance our experimental results demonstrate that the proposed mechanism dramatically reduces the computation time without sacrificing the classification accuracy for samples involving real life data sets as well as artificial data sets the results especially demonstrate the computational advantage for large data sets such as those involved in data mining and text categorization applications

explicit magnification control of self organizing maps for forbidden data in this paper we examine the scope of validity of the explicit self organizing map som magnification control scheme of bauer 1996 on data for which the theory does not guarantee success namely data that are n dimensional nges2 and whose components in the different dimensions are not statistically independent the bauer algorithm is very attractive for the possibility of faithful representation of the probability density function pdf of a data manifold or for discovery of rare events among other properties since theoretically unsupported data of higher dimensionality and higher complexity would benefit most from the power of explicit magnification control we conduct systematic simulations on forbidden data for the unsupported n 2 cases that we investigate the simulations show that even though the magnification exponent alphaachieved achieved by magnification control is not the same as the desired alphadesired alphaachieved systematically follows alphadesired with a slowly increasing positive offset we show that for simple synthetic higher dimensional data information theoretically optimum pdf matching alphaachieved 1 can be achieved and that negative magnification has the desired effect of improving the detectability of rare classes in addition we further study theoretically unsupported cases with real data

kernel based equiprobabilistic topographic map formation we introduce a new unsupervised competitive learning rule the kernel based maximum entropy learning rule kmer which performs equiprobabilistic topographic map formation in regular fixed topology lattices for use with nonparametric density estimation as well as nonparametric regression analysis the receptive fields of the formal neurons are overlapping radially symmetric kernels compatible with radial basis functions rbfs but unlike other learning schemes the radii of these kernels do not have to be chosen in an ad hoc manner the radii are adapted to the local input density together with the weight vectors that define the kernel centers so as to produce maps of which the neurons have an equal probability to be active equiprobabilistic maps both an online and a batch version of the learning rule are introduced which are applied to nonparametric density estimation and regression respectively the application envisaged is blind source separation bss from nonlinear noisy mixtures

density based clustering with topographic maps a new unsupervised competitive learning rule is introduced called the kernel based maximum entropy learning rule kmer for equiprobabilistic topographic map formation the application envisaged is density based clustering an empirical study is conducted to compare the clustering performance of kmer with that of a number of other unsupervised competitive learning rules

code vector density in topographic mappings scalar case the author derives some new results that build on his earlier work 1989 of combining vector quantization vq theory and topographic mapping tm theory a vq model with a noisy transmission medium is used to model the processes that occur in tms which leads to the standard tm training algorithm albeit with a slight modification to the encoding process to emphasize this difference the model is called a topographic vector quantizer tvq in the continuum limit of the one dimensional scalar tvq it is found that the density of code vectors is proportional to p x sup a alpha 1 3 assuming that the transmission medium introduces additive noise with a zero mean symmetric monotically decreasing probability density this result is dramatically different from the result that is predicted when the standard tm training algorithm is used with a uniform symmetric neighborhood n n and it is noted that this difference arises entirely from using minimum distortion rather than nearest neighbor encoding

meixner class of non commutative generalized stochastic processes with freely independent values ii the generating function let t be an underlying space with a non atomic measure on it in comm math phys 292 99 129 2009 the meixner class of non commutative generalized stochastic processes with freely independent values omega omega t t in t was characterized through the continuity of the corresponding orthogonal polynomials in this paper we derive a generating function for these orthogonal polynomials the first question we have to answer is what should serve as a generating function for a system of polynomials of infinitely many non commuting variables we construct a class of operator valued functions z z t t in t such that z t commutes with s for any s t in t then a generating function can be understood as g z omega sum n 0 infty int t n p n omega t 1 dots omega t n z t 1 dots z t n sigma dt 1 dots sigma dt n where p n omega t 1 dots omega t n is the kernel of the nth orthogonal polynomial we derive an explicit form of g z which has a resolvent form and resembles the generating function in the classical case albeit it involves integrals of non commuting operators we finally discuss a related problem of the action of the annihilation operators partial t t in t in contrast to the classical case we prove that the operators t related to the free gaussian and poisson processes have a property of globality this result is genuinely infinite dimensional since in one dimension one loses the notion of globality

self organizing maps for imprecise data self organizing maps soms consist of a set of neurons arranged in such a way that there are neighbourhood relationships among neurons following an unsupervised learning procedure the input space is divided into regions with common nearest neuron vector quantization allowing clustering of the input vectors in this paper we propose an extension of the soms for data imprecisely observed self organizing maps for imprecise data soms id the learning algorithm is based on two distances for imprecise data in order to illustrate the main features and to compare the performances of the proposed method we provide a simulation study and different substantive applications

codeword distribution for frequency sensitive competitive learning with one dimensional input data we study the codeword distribution for a conscience type competitive learning algorithm frequency sensitive competitive learning fscl using one dimensional input data we prove that the asymptotic codeword density in the limit of large number of codewords is given by a power law of the form q x c spl middot p x sup spl alpha where p x is the input data density and spl alpha depends on the algorithm and the form of the distortion measure to be minimized we further show that the algorithm can be adjusted to minimize any l sub p distortion measure with p ranging in 0 2

asymptotic level density in topological feature maps the kohonen algorithm entails a topology conserving mapping of an input pattern space x spl sub r sup n characterized by an a priori probability distribution p x x spl isin x onto a discrete lattice of neurons r with virtual positions w sub rspl isin x extending results obtained by ritter 1991 the authors show in the one dimensional case for an arbitrary monotonously decreasing neighborhood function h r r that the point density d w sub r of the virtual net is a polynomial function of the probability density p x with d w sub r spl sim p sup spl alpha w sub r here the distortion exponent is given by spl alpha 1 12r 3 1 6r and is determined by the normalized second moment r of the neighborhood function a gaussian neighborhood interaction is discussed and the analytical results are checked by means of computer simulations

a bayesian analysis of self organizing maps in this paper bayesian methods are used to analyze some of the properties of a special type of markov chain the forward transitions through the chain are followed by inverse transitions using bayes theorem backward through a copy of the same chain this will be called a folded markov chain if an appropriately defined euclidean error between the original input and its reconstruction via bayes theorem is minimized with respect to the choice of markov chain transition probabilities then the familiar theories of both vector quantizers and self organizing maps emerge this approach is also used to derive the theory of self supervision in which the higher layers of a multilayer network supervise the lower layers even though overall there is no external teacher

comments on on a novel unsupervised competitive learning algorithm for scalar quantization this note propose an alternative to a neural network for designing scaler quantizers proposed by van hulle and martinez ibid vol 5 p 498 501 may 1994 it also points out that the performance measure used is of limited applicability

winner relaxing self organizing maps a new family of self organizing maps the winner relaxing kohonen algorithm is introduced as a generalization of a variant given by kohonen in 1991 the magnification behavior is calculated analytically for the original variant a magnification exponent of 4 7 is derived the generalized version allows steering the magnification in the wide range from exponent 1 2 to 1 in the one dimensional case thus providing optimal mapping in the sense of information theory the winner relaxing algorithm requires minimal extra computations per learning step and is conveniently easy to implement

comparison of som point densities based on different criteria point densities of model codebook vectors in self organizing maps soms are evaluated in this article for a few one dimensional soms with finite grid lengths and a given probability density function of the input the numerically exact point densities have been computed the point density derived from the som algorithm turned out to be different from that minimizing the som distortion measure showing that the model vectors produced by the basic som algorithm in general do not exactly coincide with the optimum of the distortion measure a new computing technique based on the calculus of variations has been introduced it was applied to the computation of point densities derived from the distortion measure for both the classical vector quantization and the som with general but equal dimensionality of the input vectors and the grid respectively the power laws in the continuum limit obtained in these cases were found to be identical

the self organizing field kohonen maps many of the properties of the well known kohonen map algorithm are not easily derivable from its discrete formulation for instance the projection implemented by the map from a high dimensional input space to a lower dimensional map space must be properly regarded as a projection from a smooth manifold to a lattice and in this framework some of its properties are not easily identified this paper describes the self organizing field a continuous embedding of a smooth manifold the map into another the input manifold that implements a topological map by self organization the adaptation of the self organizing field is governed by a set of differential equations analogous to the difference equations that determine weights updates in the kohonen map this paper derives several properties of the self organizing field and shows that the emergence of certain structures on the brain like the columnar organization in the primary visual cortex arise naturally in the new model

a nonparametric approach to the truncated regression problem abstract a description is given of a new method of estimating the regression parameters in the linear regression model from data where the dependent variable is subject to truncation the residual distribution is allowed to be unspecified the method is iterative and involves estimation of the residual distribution under the truncated sampling scheme the technique can be interpreted as an iterative bias adjustment of the observations in order to correct the regression relationship in the sampled population to match that of the model a simulation study compares the performance of various estimators including one suggested by bhattacharya chernoff and yang 1983 this truncation regression problem arises in many contexts of scientific and social research in economics tobin 1958 analyzed household expenditure on durable goods using a regression model that took account of the fact that the expenditure is always nonnegative a more general situation was studied by hausman and wise 1976 1977 in conn

asymptotic level density for a class of vector quantization processes it is shown that for a class of vector quantization processes related to neural modeling that the asymptotic density q x of the quantization levels in one dimension in terms of the input signal distribution p x is a power law q x c p x sup alpha where the exponent alpha depends on the number n of neighbors on each side of a unit and is given by alpha 2 3 1 3n sup 2 3 n 1 sup 2 the asymptotic level density is calculated and monte carlo simulations are presented

asymptotic quantization error of continuous signals and the quantization dimension extensions of the limiting qnanfizafion error formula of bennet are proved these are of the form d s k n f n beta b where n is the number of output levels d s k n f is the s th moment of the metric distance between quantizer input and output beta b 0 k s beta is the signal space dimension and f is the signal distribution if a suitably well behaved k dimensional signal density f x exists b b s k int f rho x dx 1 rho rho k s k and b s k does not depend on f for k 1 s 2 this reduces to bennett s formula if f is the cantor distribution on 0 1 0 and this k equals the fractal dimension of the cantor set 12 13 random quantization optimal quantization in the presence of an output information constraint and quantization noise in high dimensional spaces are also investigated

the effect of concave and convex weight adjustments on self organizing maps two nonlinear models of weight adjustments of self organizing maps are derived to obtain desirable densities of output units one that approaches the probability distribution p spl xi of the inputs and one that approaches a uniform distribution if a convex model is used to adjust weights the density of output units can be made to approach p spl xi instead of the p spl xi sup 2 3 which results from the linear weight adjustment of kohonen s self organizing maps if a concave model of weight adjustments is used the density approaches a uniform distribution and the winner frequency distribution of output units is proportional to p spl xi the former can provide more efficient data representations for vector quantization while the latter can provide more meaningful measures for cluster analysis numerical demonstrations validate the mathematical derivations the convergence of the concave model is faster than the linear and convex models while the convergence of the convex model is comparable to that of the linear model

maximum entropy and learning theory we derive the learning theory recently reported by tishby levin and solla tls directly from the principle of maximum entropy instead of statistical mechanics the theory generally applies to any problem of modeling data we analyze an elementary example for which we find the predictions consistent with intuition and conventional statistical results and we numerically examine the more realistic problem of training a competitive net to learn a one dimensional probability density from samples the tls theory is useful for predicting average training behavior