motion based behaviour learning profiling and classification in the presence of anomalies techniques for understanding video object motion activity are becoming increasingly important with the widespread adoption of cctv surveillance systems motion trajectories provide rich spatiotemporal information about an object s activity this paper presents a novel technique for clustering and classification of motion in the proposed motion learning system trajectories are treated as time series and modelled using modified dft discrete fourier transform based coefficient feature space representation a framework iterative hsact lvq hierarchical semi agglomerative clustering learning vector quantization is proposed for learning of patterns in the presence of significant number of anomalies in training data a novel modelling technique referred to as m mediods is also proposed that models the class containing n members with m mediods once the m mediods based model for all the classes have been learnt the classification of new trajectories and anomaly detection can be performed by checking the closeness of said trajectory to the models of known classes a mechanism based on agglomerative approach is proposed for anomaly detection our proposed techniques are validated using variety of simulated and complex real life trajectory data sets

object trajectory based activity classification and recognition using hidden markov models motion trajectories provide rich spatiotemporal information about an object s activity this paper presents novel classification algorithms for recognizing object activity using object motion trajectory in the proposed classification system trajectories are segmented at points of change in curvature and the subtrajectories are represented by their principal component analysis pca coefficients we first present a framework to robustly estimate the multivariate probability density function based on pca coefficients of the subtrajectories using gaussian mixture models gmms we show that gmm based modeling alone cannot capture the temporal relations and ordering between underlying entities to address this issue we use hidden markov models hmms with a data driven design in terms of number of states and topology e g left right versus ergodic experiments using a database of over 5700 complex trajectories obtained from uci kdd data archives and columbia university multimedia group subdivided into 85 different classes demonstrate the superiority of our proposed hmm based scheme using pca coefficients of subtrajectories in comparison with other techniques in the literature

dynamics of interacting neural networks the dynamics of interacting perceptrons is solved analytically for a directed flow of information the system runs into a state which has a higher symmetry than the topology of the model a symmetry breaking phase transition is found with increasing learning rate in addition it is shown that a system of interacting perceptrons which is trained on the history of its minority decisions develops a good strategy for the problem of adaptive competition known as the bar problem or minority game

semi supervised metric learning via topology preserving multiple semi supervised assumptions learning an appropriate distance metric is a critical problem in pattern recognition this paper addresses the problem of semi supervised metric learning we propose a new regularized semi supervised metric learning rssml method using local topology and triplet constraints our regularizer is designed and developed based on local topology which is represented by local neighbors from the local smoothness cluster low density and manifold information point of view the regularizer is then combined with the large margin hinge loss on the triplet constraints in other words we keep a large margin between different labeled samples and in the meanwhile we use the unlabeled samples to regularize it then the semi supervised metric learning method is developed we have performed experiments on classification using publicly available databases to evaluate the proposed method to our best knowledge this is the only method satisfying all the three semi supervised assumptions namely smoothness cluster low density and manifold experimental results have shown that the proposed method outperforms state of the art semi supervised distance metric learning algorithms

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

shape activity a continuous state hmm for moving deforming shapes with application to abnormal activity detection the aim is to model activity performed by a group of moving and interacting objects which can be people cars or different rigid components of the human body and use the models for abnormal activity detection previous approaches to modeling group activity include co occurrence statistics individual and joint histograms and dynamic bayesian networks neither of which is applicable when the number of interacting objects is large we treat the objects as point objects referred to as landmarks and propose to model their changing configuration as a moving and deforming shape using kendall s shape theory for discrete landmarks a continuous state hidden markov model is defined for landmark shape dynamics in an activity the configuration of landmarks at a given time forms the observation vector and the corresponding shape and the scaled euclidean motion parameters form the hidden state vector an abnormal activity is then defined as a change in the shape activity model which could be slow or drastic and whose parameters are unknown results are shown on a real abnormal activity detection problem involving multiple moving objects

transient dynamics of on line learning in two layered neural networks the dynamics of on line learning in neural networks with continuous units is dominated by plateaux in the time dependence of the generalization error using tools from statistical mechanics we show for a soft committee machine the existence of several fixed points of the dynamics of learning that give rise to complicated behaviour such as cascade like runs through different plateaux with a decreasing value of the corresponding generalization error we find learning rate dependent phenomena such as splitting and disappearing of fixed points of the equations of motion the dependence of plateau lengths on the initial conditions is described analytically and simulations confirm the results

a recurrent probabilistic neural network with dimensionality reduction based on time series discriminant component analysis this paper proposes a probabilistic neural network nn developed on the basis of time series discriminant component analysis tsdca that can be used to classify high dimensional time series patterns tsdca involves the compression of high dimensional time series into a lower dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous density hidden markov model with a gaussian mixture model expressed in the reduced dimensional space the analysis can be incorporated into an nn which is named a time series discriminant component network tsdcn so that parameters of dimensionality reduction and classification can be obtained simultaneously as network coefficients according to a backpropagation through time based learning algorithm with the lagrange multiplier method the tsdcn is considered to enable high accuracy classification of high dimensional time series patterns and to reduce the computation time taken for network training the validity of the tsdcn is demonstrated for high dimensional artificial data and electroencephalogram signals in the experiments conducted during the study

video event classification and image segmentation based on noncausal multidimensional hidden markov models in this paper we propose a novel solution to an arbitrary noncausal multidimensional hidden markov model hmm for image and video classification first we show that the noncausal model can be solved by splitting it into multiple causal hmms and simultaneously solving each causal hmm using a fully synchronous distributed computing framework therefore referred to as distributed hmms next we present an approximate solution to the multiple causal hmms that is based on an alternating updating scheme and assumes a realistic sequential computing framework the parameters of the distributed causal hmms are estimated by extending the classical 1 d training and classification algorithms to multiple dimensions the proposed extension to arbitrary causal multidimensional hmms allows state transitions that are dependent on all causal neighbors we thus extend three fundamental algorithms to multidimensional causal systems i e 1 expectation maximization em 2 general forward backward gfb and 3 viterbi algorithms in the simulations we choose to limit ourselves to a noncausal 2 d model whose noncausality is along a single dimension in order to significantly reduce the computational complexity simulation results demonstrate the superior performance higher accuracy rate and applicability of the proposed noncausal hmm framework to image and video classification

learning spectral clustering with application to speech separation spectral clustering refers to a class of techniques which rely on the eigenstructure of a similarity matrix to partition points into disjoint clusters with points in the same cluster having high similarity and points in different clusters having low similarity in this paper we derive new cost functions for spectral clustering based on measures of error between a given partition and a solution of the spectral relaxation of a minimum normalized cut problem minimizing these cost functions with respect to the partition leads to new spectral clustering algorithms minimizing with respect to the similarity matrix leads to algorithms for learning the similarity matrix from fully labelled data sets we apply our learning algorithm to the blind one microphone speech separation problem casting the problem as one of segmentation of the spectrogram

notes on particular symmetric polynomials with applications this paper presents some applications using several properties of three important symmetric polynomials elementary symmetric polynomials complete symmetric polynomials and the power sum symmetric polynomials the applications includes a simple proof of el mikkawy conjecture in m e a el mikkawy appl math comput 146 2003 759 769 and a very easy proof of the newton girard formula in addition a generalization of stirling numbers is obtained

warped k means an algorithm to cluster sequentially distributed data many devices generate large amounts of data that follow some sort of sequentiality e g motion sensors e pens eye trackers etc and often these data need to be compressed for classification storage and or retrieval tasks traditional clustering algorithms can be used for this purpose but unfortunately they do not cope with the sequential information implicitly embedded in such data thus we revisit the well known k means algorithm and provide a general method to properly cluster sequentially distributed data we present warped k means wkm a multi purpose partitional clustering procedure that minimizes the sum of squared error criterion while imposing a hard sequentiality constraint in the classification step we illustrate the properties of wkm in three applications one being the segmentation and classification of human activity wkm outperformed five state of the art clustering techniques to simplify data trajectories achieving a recognition accuracy of near 97 which is an improvement of around 66 over their peers moreover such an improvement came with a reduction in the computational cost of more than one order of magnitude

learning a mahalanobis distance metric for data clustering and classification distance metric is a key issue in many machine learning algorithms this paper considers a general problem of learning from pairwise constraints in the form of must links and cannot links as one kind of side information a must link indicates the pair of the two data points must be in a same class while a cannot link indicates that the two data points must be in two different classes given must link and cannot link information our goal is to learn a mahalanobis distance metric under this metric we hope the distances of point pairs in must links are as small as possible and those of point pairs in cannot links are as large as possible this task is formulated as a constrained optimization problem in which the global optimum can be obtained effectively and efficiently finally some applications in data clustering interactive natural image segmentation and face pose estimation are given in this paper experimental results illustrate the effectiveness of our algorithm

learning in two layered networks with correlated examples on line learning in layered perceptrons is often hampered by plateaus in the time dependence of the performance studies on backpropagation in networks with a small number of input units have revealed that correlations between subsequently presented patterns shorten the length of such plateaus we show how to extend the statistical mechanics framework to quantitatively check the effect of correlations on learning in networks with a large number of input units the surprisingly compact description we obtain makes it possible to derive properties of on learning with correlations directly from studies on on line learning without correlations

view invariant representation and recognition of actions analysis of human perception of motion shows that information for representing the motion is obtained from the dramatic changes in the speed and direction of the trajectory in this paper we present a computational representation of human action to capture these dramatic changes using spatio temporal curvature of 2 d trajectory this representation is compact view invariant and is capable of explaining an action in terms of meaningful action units called dynamic instants and intervals a dynamic instant is an instantaneous entity that occurs for only one frame and represents an important change in the motion characteristics an interval represents the time period between two dynamic instants during which the motion characteristics do not change starting without a model we use this representation for recognition and incremental learning of human actions the proposed method can discover instances of the same action performed by different people from different view points experiments on 47 actions performed by 7 individuals in an environment with no constraints shows the robustness of the proposed method

discriminant adaptive nearest neighbor classification nearest neighbour classification expects the class conditional probabilities to be locally constant and suffers from bias in high dimensions we propose a locally adaptive form of nearest neighbour classification to try to ameliorate this curse of dimensionality we use a local linear discriminant analysis to estimate an effective metric for computing neighbourhoods we determine the local decision boundaries from centroid information and then shrink neighbourhoods in directions orthogonal to these local decision boundaries and elongate them parallel to the boundaries thereafter any neighbourhood based classifier can be employed using the modified neighbourhoods the posterior probabilities tend to be more homogeneous in the modified neighbourhoods we also propose a method for global dimension reduction that combines local dimension information in a number of examples the methods demonstrate the potential for substantial improvements over nearest neighbour classification

natural gradient learning algorithms for rbf networks radial basis function rbf networks are one of the most widely used models for function approximation and classification there are many strange behaviors in the learning process of rbf networks such as slow learning speed and the existence of the plateaus the natural gradient learning method can overcome these disadvantages effectively it can accelerate the dynamics of learning and avoid plateaus in this letter we assume that the probability density function pdf of the input and the activation function are gaussian first we introduce natural gradient learning to the rbf networks and give the explicit forms of the fisher information matrix and its inverse second since it is difficult to calculate the fisher information matrix and its inverse when the numbers of the hidden units and the dimensions of the input are large we introduce the adaptive method to the natural gradient learning algorithms finally we give an explicit form of the adaptive natural gradient learning algorithm and compare it to the conventional gradient descent method simulations show that the proposed adaptive natural gradient method which can avoid the plateaus effectively has a good performance when rbf networks are used for nonlinear functions approximation

learning and generalization in a two layer neural network the role of the vapnik chervonvenkis dimension bounds for the generalization ability of neural networks based on vapnik chervonenkis vc theory are compared with statistical mechanics results for the case of the parity machine for fixed phase space dimension the vc dimension can grows arbitrarily by increasing the number k of hidden units generalization is impossible up to a critical number of training examples that grows with the vc dimension the asymptotic decrease of the generalization error mathrm ensuremath varepsilon mathit g comes out independent of k and the vc bounds strongly overestimate mathrm ensuremath varepsilon mathit g this shows that phase space dimension and vc dimension can play independent and different roles for the generalization process

semantic preserving distance metric learning and applications how do we accurately browse a large set of images or efficiently annotate the images from an image library image clustering methods are invaluable tools for applications such as content based image retrieval and image annotation to perform these tasks it is critical to have proper features to describe the visual and semantic content of images and to define an accurate distance metric to measure the dissimilarity between any two images however existing methods which adopt the features of color histograms edge direction histograms and shape context lack the ability to describe semantic content to solve this problem we propose a new approach that utilizes user provided pairwise constraints to describe the semantic relationship between two images a semantic preserving distance metric learning sp dml algorithm is developed to explore the complementary characteristics of the visual features and pairwise constraints in a unified feature space in this space the learned distance metric can be used to measure the dissimilarity between two images specifically the manifold structure adopted in sp dml is revealed by the image s visual features to integrate semantic contents in distance metric learning sp dml utilizes pairwise constraints to build semantic patches and align these patches to obtain the optimal distance metric for the new feature space experimental results in image clustering demonstrate that the performance of sp dml is appealing

learning dynamics on different timescales the special character of certain degrees of freedom in two layered neural networks is investigated for on line learning of realizable rules our analysis shows that the dynamics of these degrees of freedom can be put on a faster timescale than those remaining with the profit of speeding up the overall adaptation process this is shown for two groups of degrees of freedom second layer weights and bias weights for the former case our analysis provides a theoretical explanation of phenomenological findings the resulting learning algorithm is compared with natural gradient descent in order to check whether the proposed scaling can be naturally derived from that type of learning rule

learning a rule in a multilayer neural network the problem of learning from examples in multilayer networks is studied within the framework of statistical mechanics using the replica formalism we calculate the average generalization error of a fully connected committee machine in the limit of a large number of hidden units if the number of training examples is proportional to the number of inputs in the network the generalization error as a function of the training set size approaches a finite value if the number of training examples is proportional to the number of weights in the network we find first order phase transitions with a discontinuous drop in the generalization error for both binary and continuous weights

statistical dynamics of on line independent component analysis the learning dynamics of on line independent component analysis is analysed in the limit of large data dimension we study a simple hebbian learning algorithm that can be used to separate out a small number of non gaussian components from a high dimensional data set the de mixing matrix parameters are confined to a stiefel manifold of tall orthogonal matrices and we introduce a natural gradient variant of the algorithm which is appropriate to learning on this manifold for large input dimension the parameter trajectory of both algorithms passes through a sequence of unstable fixed points each described by a diffusion process in a polynomial potential choosing the learning rate too large increases the escape time from each of these fixed points effectively trapping the learning in a sub optimal state in order to avoid these trapping states a very low learning rate must be chosen during the learning transient resulting in learning time scales of o n2 or o n3 iterations where n is the data dimension escape from each sub optimal state results in a sequence of symmetry breaking events as the algorithm learns each source in turn this is in marked contrast to the learning dynamics displayed by related on line learning algorithms for multilayer neural networks and principal component analysis although the natural gradient variant of the algorithm has nice asymptotic convergence properties it has an equivalent transient dynamics to the standard hebbian algorithm

generalization ability of perceptrons with continuous outputs in this paper we examine the influence of different input output relations on the generalization ability of a single layer perceptron the input output relations can be linear binary or sigmoid with this choice we take into account most of the cases which are of present interest the generalization problem will be realizable or unrealizable if the input output relations for teacher and student are identical or not we show that sometimes it can have a positive effect on the generalization ability if one learns with errors

kernel based representation for 2d 3d motion trajectory retrieval and classification this paper proposes a novel kernel space representation for motion trajectories contrasted to most trajectory representation methods in the literature our method is more generic in the sense that it can be applied to either 2d or 3d trajectories in the proposed method a trajectory is firstly projected by the kernel principal component analysis kpca which can be considered as an implicit mapping to a much higher dimensional feature space the high dimensionality can effectively improve the accuracy in recognizing motion trajectories then nonparametric discriminant analysis nda is used to extract the most discriminative features from the kpca feature space the synergistic effect of kpca and nda leads to better class separability and makes the proposed trajectory representation a more powerful discriminator the experimental validation of the proposed method is conducted on the australian sign language asl data set the results show that our method performs significantly better in both trajectory classification and retrieval than the state of the art techniques

on line learning in the ising perceptron on line learning of both binary and continuous rules in an ising space is studied learning is achieved by using an artificial parameter a weight vector which is constrained to the surface of a hypersphere spherical constraint in the case of a binary rule the generalization error decays to zero super exponentially as exp c 2 where is the number of examples divided by n the size of the input vector and c 0 much faster learning is obtained in the case of continuous activation functions where the generalization error decays as exp e the number of steps required for perfect learning is estimated for both scenarios and compared with simulations

on line learning of a mixture of experts neural network the on line learning of a mixture of experts system is studied in the framework of statistical physics the time dependence of the overlap order parameters during training is calculated analytically in the thermodynamic limit when the number of training examples is small each expert is in a symmetric state as the number of time steps approaches a critical point the symmetric state begins to disintegrate this symmetry breaking behaviour is accounted for by means of a gating network in the symmetric state the gating network has little effect on the learning but when the symmetry is broken the gating network assigns the experts to appropriate subspaces in the input space a generalization curve shows a plateau between the symmetric and broken symmetry states we also find that the learning curves show different behaviours depending on the stiffness of the gating function

perfect loss of generalization due to noise in k 2 parity machines learning in a specific type of multilayer network referred to as a k 2 parity machine is studied in the limit that both the system size n and the number of examples m become infinite while the ratio alpha m n remains finite the machine consists of k 2 hidden units with non overlapping receptive fields each of size n 2 the output is the sign of the product of the two hidden units for each input we investigate incremental learning by empirically using a least action algorithm in the following two learning paradigms in the first it is assumed that each example is transmitted perfectly to a student we show that an ability to generalize emerges as the rescaled length of the connection vector l reaches a critical value lc further we show that a student can identify the target exactly in the limit alpha to infinity where the prediction error epsilon decreases to zero as epsilon approximately 0 441 alpha 13 in the second paradigm we examine what happens if each teacher signal is reversed to the opposite sign at a noise rate l for small lambda it is found that the prediction error converges to a finite value of o square root lambda in o lambda 32 iterations however for a noise rate beyond a critical value lambda c approximately 0 175 the student cannot acquire any generalization ability even as alpha to infinity

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

on line learning with restricted training sets an exactly solvable case we solve the dynamics of on line hebbian learning in large perceptrons exactly for the regime where the size of the training set scales linearly with the number of inputs we consider both noiseless and noisy teachers our calculation cannot be extended to non hebbian rules but the solution provides a convenient and welcome benchmark with which to test more general and advanced theories for solving the dynamics of learning with restricted training sets

learning by on line gradient descent we study on line gradient descent learning in multilayer networks analytically and numerically the training is based on randomly drawn inputs and their corresponding outputs as defined by a target rule in the thermodynamic limit we derive deterministic differential equations for the order parameters of the problem which allow an exact calculation of the evolution of the generalization error first we consider a single layer perceptron with sigmoidal activation function learning a target rule defined by a network of the same architecture for this model the generalization error decays exponentially with the number of training examples if the learning rate is sufficiently small however if the learning rate is increased above a critical value perfect learning is no longer possible for architectures with hidden layers and fixed hidden to output weights such as the parity and the committee machine we find additional effects related to the existence of symmetries in these problems

frameworks for multivariate m mediods based modeling and classification in euclidean and general feature spaces this paper presents an extension of m mediods based modeling technique to cater for multimodal distributions of sample within a pattern the classification of new samples and anomaly detection is performed using a novel classification algorithm which can handle patterns with underlying multivariate probability distributions we have proposed two frameworks namely mmc es and mmc gfs to enable our proposed multivarite m mediods based modeling and classification approach workable for any feature space with a computable distance metric mmc es framework is specialized for finite dimensional features in euclidean space whereas mmc gfs works on any feature space with a computable distance metric experimental results using simulated and complex real life dataset show that multivariate m mediods based frameworks are effective and give superior performance than competitive modeling and classification techniques especially when the patterns exhibit multivariate probability density functions

distance metric learning for large margin nearest neighbor classification the accuracy of k nearest neighbor knn classification depends significantly on the metric used to compute distances between different examples in this paper we show how to learn a mahalanobis distance metric for knn classification from labeled examples the mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes knn classification using euclidean distances in our approach the metric is trained with the goal that the k nearest neighbors always belong to the same class while examples from different classes are separated by a large margin as in support vector machines svms the margin criterion leads to a convex optimization based on the hinge loss unlike learning in svms however our approach requires no modification or extension for problems in multiway as opposed to binary classification in our framework the mahalanobis distance metric is obtained as the solution to a semidefinite program on several data sets of varying size and difficulty we find that metrics trained in this way lead to significant improvements in knn classification sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster we show how to learn and combine these local metrics in a globally integrated manner

on the efficient evaluation of probabilistic similarity functions for image retrieval probabilistic approaches are a promising solution to the image retrieval problem that when compared to standard retrieval methods can lead to a significant gain in retrieval accuracy however this occurs at the cost of a significant increase in computational complexity in fact closed form solutions for probabilistic retrieval are currently available only for simple probabilistic models such as the gaussian or the histogram we analyze the case of mixture densities and exploit the asymptotic equivalence between likelihood and kullback leibler kl divergence to derive solutions for these models in particular 1 we show that the divergence can be computed exactly for vector quantizers vqs and 2 has an approximate solution for gauss mixtures gms that in high dimensional feature spaces introduces no significant degradation of the resulting similarity judgments in both cases the new solutions have closed form and computational complexity equivalent to that of standard retrieval approaches

a new lda based face recognition system which can solve the small sample size problem a new lda based face recognition system is presented in this paper linear discriminant analysis lda is one of the most popular linear projection techniques for feature extraction the major drawback of applying lda is that it may encounter the small sample size problem in this paper we propose a new lda based technique which can solve the small sample size problem we also prove that the most expressive vectors derived in the null space of the within class scatter matrix using principal component analysis pca are equal to the optimal discriminant vectors derived in the original space using lda the experimental results show that the new lda process improves the performance of a face recognition system signi cantly 2000 pattern recognition society published by elsevier science ltd all rights reserved

the dynamics of on line principal component analysis the learning dynamics of an on line algorithm for principal component analysis is described exactly in the thermodynamic limit by means of coupled ordinary differential equations for a set of order parameters it is demonstrated that learning is delayed significantly because existing symmetries among student vectors have to be broken a closely related effect is the perfect or partial loss of initial knowledge in the course of learning the analysis shows that different learning rates for the student vectors improve the performance of the algorithm drastically

rapid and brief communication extending the relevant component analysis algorithm for metric learning using both positive and negative equivalence constraints relevant component analysis rca is a recently proposed metric learning method for semi supervised learning applications it is a simple and efficient method that has been applied successfully to give impressive results however rca can make use of supervisory information in the form of positive equivalence constraints only in this paper we propose an extension to rca that allows both positive and negative equivalence constraints to be incorporated experimental results show that the extended rca algorithm is effective

on the realizability of the critical points of a realizable list abstract the nonnegative inverse eigenvalue problem niep is to characterize the spectra of entrywise nonnegative matrices a finite multiset of complex numbers is called realizable if it is the spectrum of an entrywise nonnegative matrix monov conjectured that the kth moments of the list of critical points of a realizable list are nonnegative johnson further conjectured that the list of critical points must be realizable in this work johnson s conjecture and consequently monov s conjecture is established for a variety of important cases including ciarlet spectra sule manova spectra spectra realizable via companion matrices and spectra realizable via similarity by a complex hadamard matrix additionally we prove a result on differentiators and trace vectors and use it to provide an alternative proof of a result due to malamud and a generalization of a result due to kushel and tyaglov on circulant matrices implications for further research are discussed

learning choquet integral based metrics for semisupervised clustering we consider an application of fuzzy measures to the problem of metric learning in semisupervised clustering we investigate the necessary and sufficient conditions on the underlying fuzzy measure that make the discrete choquet integral suitable for defining a metric as a byproduct we can obtain the analogous conditions for the ordered weighted averaging owa operators which constitute a special case we then generalize these results for power based choquet and owa operators we show that this metric learning problem can be formulated as a linear programming problem and specify the required sets of linear constraints we present the results of numerical experiments on artificial and real world datasets which illustrate the potential usefulness and limitations of this construction

human action recognition in unconstrained videos by explicit motion modeling human action recognition in unconstrained videos is a challenging problem with many applications most state of the art approaches adopted the well known bag of features representations generated based on isolated local patches or patch trajectories where motion patterns such as object object and object background relationships are mostly discarded in this paper we propose a simple representation aiming at modeling these motion relationships we adopt global and local reference points to explicitly characterize motion information so that the final representation is more robust to camera movements which widely exist in unconstrained videos our approach operates on the top of visual codewords generated on dense local patch trajectories and therefore does not require foreground background separation which is normally a critical and difficult step in modeling object relationships through an extensive set of experimental evaluations we show that the proposed representation produces a very competitive performance on several challenging benchmark data sets further combining it with the standard bag of features or fisher vector representations can lead to substantial improvements

perturbations on constructible lists preserving realizability in the niep and questions of monov abstract in 5 the authors showed that if 1 2 2 4 n is realizable where 1 is the perron eigenvalue and 2 is non real then so too is 1 4 t 2 t 2 t 4 n they asked if 4 can be replaced by 1 or 2 or what is the least possible multiple c 1 of t in order for this perturbation to be realizable in 2 the authors showed that for n 4 one can find certain spectra for which the result holds when c 1 provided t is small in this paper we show that c 2 is best possible and we construct a realizing matrix for c 2 when t is sufficiently small we also address some questions of monov concerning the realizability of the derivative of a realizable polynomial and if such a polynomial must have positive power sums

neural gas network for vector quantization and its application to time series prediction a neural network algorithm based on a soft max adaptation rule is presented this algorithm exhibits good performance in reaching the optimum minimization of a cost function for vector quantization data compression the soft max rule employed is an extension of the standard k means clustering procedure and takes into account a neighborhood ranking of the reference weight vectors it is shown that the dynamics of the reference weight vectors during the input driven adaptation procedure are determined by the gradient of an energy function whose shape can be modulated through a neighborhood determining parameter and resemble the dynamics of brownian particles moving in a potential determined by the data point density the network is used to represent the attractor of the mackey glass equation and to predict the mackey glass time series with additional local linear mappings for generating output values the results obtained for the time series prediction compare favorably with the results achieved by backpropagation and radial basis function networks

weight decay induced phase transitions in multilayer neural networks we investigate layered neural networks with differentiable activation function and student vectors without normalization constraint by means of equilibrium statistical physics we consider the learning of perfectly realizable rules and find that the length of student vectors becomes infinite unless a proper weight decay term is added to the energy then the system undergoes a first order phase transition between states with very long student vectors and states where the lengths are comparable to those of the teacher vectors additionally in both configurations there is a phase transition between a specialized and an unspecialized phase an anti specialized phase with long student vectors exists in networks with a small number of hidden units

modeling geometric temporal context with directional pyramid co occurrence for action recognition in this paper we present a new geometric temporal representation for visual action recognition based on local spatio temporal features first we propose a modified covariance descriptor under the log euclidean riemannian metric to represent the spatio temporal cuboids detected in the video sequences compared with previously proposed covariance descriptors our descriptor can be measured and clustered in euclidian space second to capture the geometric temporal contextual information we construct a directional pyramid co occurrence matrix dpcm to describe the spatio temporal distribution of the vector quantized local feature descriptors extracted from a video dpcm characterizes the co occurrence statistics of local features as well as the spatio temporal positional relationships among the concurrent features these statistics provide strong descriptive power for action recognition to use dpcm for action recognition we propose a directional pyramid co occurrence matching kernel to measure the similarity of videos the proposed method achieves the state of the art performance and improves on the recognition performance of the bag of visual words bovws models by a large margin on six public data sets for example on the kth data set it achieves 98 78 accuracy while the bovw approach only achieves 88 06 on both weizmann and ucf cil data sets the highest possible accuracy of 100 is achieved

active selection of clustering constraints this paper examines active selection of clustering constraints which has become a topic of significant interest in constrained clustering active selection of clustering constraints which is known as minimizing the cost of acquiring constraints also includes quantifying utility of a given constraint set a sequential method is proposed in this paper to select the most beneficial set of constraints actively the proposed method uses information of boundary points and transition regions extracted by data description methods to introduce a utility measure for constraints since previously selected constraints affect the utility of remaining candidate constraints a method is proposed to update the utility of remaining constraints after selection of each constraint experiments carried out on synthetic and real datasets show that the proposed method improves the accuracy of clustering while reducing human interaction highlightsactive constraint selection as an open issue in constrained clustering is examined the method relies on the information of boundaries and transition regions of data a utility measure based on the description of data is proposed for constraints this utility measure considers the constraint utility dependencies constraints are selected in a sequential approach utilizing the utility measure

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

inverse eigenvalue problems a collection of inverse eigenvalue problems are identified and classified according to their characteristics current developments in both the theoretic and the algorithmic aspects are summarized and reviewed in this paper this exposition also reveals many open questions that deserve further study an extensive bibliography of pertinent literature is attached

stochastic trapping in a solvable model of on line independent component analysis previous analytical studies of on line independent component analysis ica learning rules have focused on asymptotic stability and efficiency in practice the transient stages of learning are often more significant in determining the success of an algorithm this is demonstrated here with an analysis of a hebbian ica algorithm which can find a small number of nongaussian components given data composed of a linear mixture of independent source signals an idealized data model is considered in which the sources comprise a number of nongaussian and gaussian sources and a solution to the dynamics is obtained in the limit where the number of gaussian sources is infinite previous stability results are confirmed by expanding around optimal fixed points where a closed form solution to the learning dynamics is obtained however stochastic effects are shown to stabilize otherwise unstable suboptimal fixed points conditions required to destabilize one such fixed point are obtained for the case of a single nongaussian component indicating that the initial learning rate required to escape successfully is very low o n 2 where n is the data dimension resulting in very slow learning typically requiring o n3 iterations simulations confirm that this picture holds for a finite system

stochastic dynamics of supervised learning the stochastic evolution of adiabatic slow backpropagation training of a neural network is discussed and a fokker planck equation for the post training distribution function in the network space is derived the distribution obtained differs from the one given by radons et al 1990 studying the character of the post training distribution the authors find that except under very special circumstances the distribution will be non gibbsian the validity of the present approach is tested on a simple backpropagation learning system in one dimension which can be solved analytically as well implications of the fokker planck approach for general situations are examined in the local linear approximation surprisingly they find that the post training distribution is isotropic close to its peak hence simpler than the corresponding gibbs distribution

twin mahalanobis distance based support vector machines for pattern recognition twin support vector machines tsvms achieve fast training speed and good performance for data classification however tsvms do not take full advantage of the statistical information in data such as the covariance of each class of data this paper proposes a new twin mahalanobis distance based support vector machine tmsvm classifier in which two mahalanobis distance based kernels are constructed according to the covariance matrices of two classes of data for optimizing the nonparallel hyperplanes tmsvms have a special case of tsvms when the covariance matrices in a reproducing kernel hilbert space are degenerated to the identity ones tmsvms are suitable for many real problems especially for the case that the covariance matrices of two classes of data are obviously different the experimental results on several artificial and benchmark datasets indicate that tmsvms not only possess fast learning speed but also obtain better generalization than tsvms and other methods

a family of symmetric polynomials of the eigenvalues of a matrix in this paper we consider a family of symmetric polynomials of the eigenvalues of a complex matrix a and find an explicit expression of each member of the family as a polynomial of the entries of a with positive coefficients in the case of a nonnegative matrix one immediately obtains a family of inequalities involving matrix eigenvalues and diagonal entries equivalent forms of some of the obtained results as well as connections with known results and specific applications are also presented in the concluding part of the paper we provide comments and conjecture further inequalities related with nonnegative matrices

dynamics of learning near singularities in layered networks we explicitly analyze the trajectories of learning near singularities in hierarchical networks such as multilayer perceptrons and radial basis function networks which include permutation symmetry of hidden nodes and show their general properties such symmetry induces singularities in their parameter space where the fisher information matrix degenerates and odd learning behaviors especially the existence of plateaus in gradient descent learning arise due to the geometric structure of singularity we plot dynamic vector fields to demonstrate the universal trajectories of learning near singularities the singularity induces two types of plateaus the on singularity plateau and the near singularity plateau depending on the stability of the singularity and the initial parameters of learning the results presented in this letter are universally applicable to a wide class of hierarchical models detailed stability analysis of the dynamics of learning in radial basis function networks and multilayer perceptrons will be presented in separate work