simultaneous parameter tracking and state estimation in a linear system abstract the task of simultaneous tracking of time varying parameters and estimation of the state is treated for a linear system described by a time varying input output armax or delta model with known c noise parameters first a bayesian approach based conceptual solution is presented then it is shown that utilizing the properties of the observer canonical state model algebraic recursion operating on the joint parameter and state mean and covariance matrix can be obtained with no approximation involved several illustrative examples are included

stabilized least squares type adaptive identifiers least squares type adaptive identifiers are proposed which are stabilized in the sense that the covariance equation has a positive definition asymptotically stable equilibrium this keeps the adaptive gain matrix bounded and the adaptive abilities of the algorithm always properly in operation the convergence speed of the stabilized adaptive identifiers is proportional to the signal level when the signal level is low and can be made to achieve almost arbitrarily fast convergence for high signal levels the algorithms require the inversion of positive definite matrices and or the solution of associated linear equations which is numerically well behaved as a result a slightly increased computational complexity as compared to known least squares type algorithms is to be paid for advanced stability and adaptive properties of the algorithms

bm e discriminative density propagation for visual tracking we introduce bm3e a conditional bayesian mixture of experts markov model that achieves consistent probabilistic estimates for discriminative visual tracking the model applies to problems of temporal and uncertain inference and represents the unexplored bottom up counterpart of pervasive generative models estimated with kalman filtering or particle filtering instead of inverting a nonlinear generative observation model at runtime we learn to cooperatively predict complex state distributions directly from descriptors that encode image observations typically bag of feature global image histograms or descriptors computed over regular spatial grids these are integrated in a conditional graphical model in order to enforce temporal smoothness constraints and allow a principled management of uncertainty the algorithms combine sparsity mixture modeling and nonlinear dimensionality reduction for efficient computation in high dimensional continuous state spaces the combined system automatically self initializes and recovers from failure the research has three contributions 1 we establish the density propagation rules for discriminative inference in continuous temporal chain models 2 we propose flexible supervised and unsupervised algorithms to learn feed forward multivalued contextual mappings multimodal state distributions based on compact conditional bayesian mixture of experts models and 3 we validate the framework empirically for the reconstruction of 3d human motion in monocular video sequences our tests on both real and motion capture based sequences show significant performance gains with respect to competing nearest neighbor regression and structured prediction methods

assessing the true role of coauthors in the h index measure of an author scientific impact abstract a method based on the classical principal component analysis leads to demonstrate that the role of co authors should give a h index measure to a group leader higher than usually accepted the method rather easily gives what is usually searched for i e an estimate of the role or weight of co authors as the additional value to an author papers popularity the construction of the co authorship popularity h matrix is exemplified and the role of eigenvalues and the main eigenvector component are discussed examples illustrate the points and serve as the basis for suggesting a generally practical application of the concept

learn mf a random subspace approach for the missing feature problem we introduce learn mf an ensemble of classifiers based algorithm that employs random subspace selection to address the missing feature problem in supervised classification unlike most established approaches learn mf does not replace missing values with estimated ones and hence does not need specific assumptions on the underlying data distribution instead it trains an ensemble of classifiers each on a random subset of the available features instances with missing values are classified by the majority voting of those classifiers whose training data did not include the missing features we show that learn mf can accommodate substantial amount of missing data and with only gradual decline in performance as the amount of missing data increases we also analyze the effect of the cardinality of the random feature subsets and the ensemble size on algorithm performance finally we discuss the conditions under which the proposed approach is most effective

mixture densities maximum likelihood and the em algorithm the problem of estimating the parameters which determine a mixture density has been the subject of a large diverse body of literature spanning nearly ninety years during the last two decades the

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

coupled prediction classification for robust visual tracking this paper addresses the problem of robust template tracking in image sequences our work falls within the discriminative framework in which the observations at each frame yield direct probabilistic predictions of the state of the target our primary contribution is that we explicitly address the problem that the prediction accuracy for different observations varies and in some cases can be very low to this end we couple the predictor to a probabilistic classifier which when trained can determine the probability that a new observation can accurately predict the state of the target that is determine the relevance or reliability of the observation in question in the particle filtering framework we derive a recursive scheme for maintaining an approximation of the posterior probability of the state in which multiple observations can be used and their predictions moderated by their corresponding relevance in this way the predictions of the relevant observations are emphasized while the predictions of the irrelevant observations are suppressed we apply the algorithm to the problem of 2d template tracking and demonstrate that the proposed scheme outperforms classical methods for discriminative tracking both in the case of motions which are large in magnitude and also for partial occlusions

mixture of regression models with varying mixing proportions a semiparametric approach in this article we study a class of semiparametric mixtures of regression models in which the regression functions are linear functions of the predictors but the mixing proportions are smoothing functions of a covariate we propose a one step backfitting estimation procedure to achieve the optimal convergence rates for both regression parameters and the nonparametric functions of mixing proportions we derive the asymptotic bias and variance of the one step estimate and further establish its asymptotic normality a modified expectation maximization type em type estimation procedure is investigated we show that the modified em algorithms preserve the asymptotic ascent property numerical simulations are conducted to examine the finite sample performance of the estimation procedures the proposed methodology is further illustrated via an analysis of a real dataset

c ondensation conditional density propagation forvisual tracking the problem of tracking curves in dense visual clutter is challenging kalman filtering is inadequate because it is based on gaussian densities which being unimo dal cannot represent simultaneous alternative hypotheses the condensation algorithm uses factored sampling previously applied to the interpretation of static images in which the probability distribution of possible interpretations is represented by a randomly generated set condensation uses learned dynamical models together with visual observations to propagate the random set over time the result is highly robust tracking of agile motion notwithstanding the use of stochastic methods the algorithm runs in near real time

effects of publications in proceedings on the measure of the core size of coauthors abstract coauthors ca of a lead investigator li can receive a rank r according to their importance in having published joint publications with the li it is commonly accepted without any proof that publications in peer review journals and for example conference proceedings do not have the same value in a cv and the same applies to papers contributing to encyclopedia and book chapters it is examined here whether the relationship between the number j of publications of some scientist with her his coauthors ranked according to their decreasing importance i e j 1 r as found by ausloos 2013 1 still holds if the overall publication list is broken into such specific types of publications several authors with different careers but mainly having worked in the field of statistical mechanics are studied here to sort out answers to the questions the exponent turns out to be weakly scientist dependent only if the maximum value of j and r is large and is 1 then the m a core value i e the core number of cas for proceedings only is about half of the total one i e when all publications are counted contributions to the numerical values from both encyclopedia and book chapters are marginal the role of a time span on m a is also examined in two cases in relation to career activity considerations it can be considered that the findings serve as a contrasting point of view on how to quantify an individual publication career as recently done by petersen et al 2010 2012 2011 2 3 4 here emphasizing the collaboration size and evolution rather than a citation count moreover specifying the type of publication through the various m a s one can distinguish different behavior patterns of a scientific publication with cas

black scholes schrodinger zipf mandelbrot model framework for improving a study of the coauthor core score abstract the data and findings by miskiewicz 2013 on the relationship between the number j of publications ranked according to their decreasing importance for some scientist with her his coauthors ca i e j 1 r as found in ausloos 2013 when specific types of publications i e proceedings in a generalized sense and peer review journals are considered are reexamined along the zipf mandelbrot law mandelbrot 1977 i e j 1 r the statistics are much improved the exponent and are compared the m a core value i e the core number of cas 1 is unaffected of course a black scholes schrodinger model framework is proposed to describe the findings

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

multivariable adaptive control identifiable parameterizations and parameter convergence the paper presents a direct model reference adaptive control algorithm for multivariable systems that has guaranteed parameter convergence properties the algorithm requires fewer parameters than existing schemes and the parameters converge to their nominal values under persistency of excitation conditions simple frequency domain conditions on the external inputs are derived to satisfy these conditions the convergence of the controller parameters to their nominal values guarantees a certain degree of robustness of the algorithms and ensures that the model matching objective is satisfied asymptotically

generalized darting monte carlo one of the main shortcomings of markov chain monte carlo samplers is their inability to mix between modes of the target distribution in this paper we show that advance knowledge of the location of these modes can be incorporated into the mcmc sampler by introducing mode hopping moves that satisfy detailed balance the proposed sampling algorithm explores local mode structure through local mcmc moves e g diffusion or hybrid monte carlo but in addition also represents the relative strengths of the different modes correctly using a set of global moves this mode hopping mcmc sampler can be viewed as a generalization of the darting method 1 we illustrate the method on learning markov random fields and evaluate it against the spherical darting algorithm on a real world vision application of inferring 3d human body pose distributions from 2d image information

comparison of approximate methods for handling hyperparameters i examine two approximate methods for computational implementation of bayesian hierarchical models that is models that include unknown hyperparameters such as regularization constants and noise levels in the evidence framework the model parameters are integrated over and the resulting evidence is maximized over the hyperparameters the optimized hyperparameters are used to define a gaussian approximation to the posterior distribution in the alternative map method the true posterior probability is found by integrating over the hyperparameters the true posterior is then maximized over the model parameters and a gaussian approximation is made the similarities of the two approaches and their relative merits are discussed and comparisons are made with the ideal hierarchical bayesian solution in moderately ill posed problems integration over hyperparameters yields a probability distribution with a skew peak which causes signifi cant biases to arise in the map method in contrast the evidence fram

on duality of regularized exponential and linear forgetting abstract regularized stabilized versions of exponential and linear forgetting in parameter tracking are shown to be dual to each other both are derived by solving basically the same bayesian decision problem where kullback leibler divergence is used to measure quasi distance between probability distributions of estimated parameters the type of forgetting depends solely on the order of arguments in kullback leibler divergence this general view indicates under which conditions one technique is superior to the other applied to the case of arx models the approach results in a class of regularized or stabilized forgetting strategies that are naturally robust with respect to poor system excitation

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

brief robust adaptive identification of slowly time varying parameters with bounded disturbances in this paper the robustness limitations of current recursive identification algorithms are highlighted then it is shown how a particular non recursive identification algorithm may improve the robustness to bounded disturbances and slowly time varying parameters at the expense of performing an on line eigenvalue decomposition on a symmetric semidefinite positive matrix furthermore contrary to robustified recursive identification algorithms this algorithm does not require any a priori knowledge of a bound on the disturbances and of a bound on the unknown parameter values

a globally convergent adaptive pole placement algorithm without a persistency of excitation requirement this paper presents an indirect adaptive control scheme for deterministic plants which are not necessarily minimum phase global convergence is established for the scheme in the sense that the closed loop poles are asymptotically assigned for the given data sequence and the system input and output remain bounded for all time a key feature of the scheme is that no persistency of excitation condition is required the algorithm uses recursive least squares with variable forgetting factor normalized regression vectors and a matrix gain with constant trace

the concave convex procedure the concave convex procedure cccp is a way to construct discrete time iterative dynamical systems that are guaranteed to decrease global optimization and energy functions monotonically this procedure can be applied to almost any optimization problem and many existing algorithms can be interpreted in terms of it in particular we prove that all expectation maximization algorithms and classes of legendre minimization and variational bounding algorithms can be reexpressed in terms of cccp we show that many existing neural network and mean field theory algorithms are also examples of cccp the generalized iterative scaling algorithm and sinkhorn s algorithm can also be expressed as cccp by changing variables cccp can be used both as a new way to understand and prove the convergence of existing optimization algorithms and as a procedure for generating new algorithms

modified least squares algorithm incorporating exponential resetting and forgetting in this paper we present the general analysis of a class of least squares algorithms with emphasis on their dynamic performance particularly in the presence of poor excitation the analysis is carried out in a deterministic framework and stresses geometrical interpretations the core of this paper is the proposal and analysis of a new algorithm which incorporates exponential forgetting and resetting to an unprejudiced treatment of data when excitation is poor the algorithm is particularly suitable for tracking time varying parameters and is similar in computational complexity to the standard recursive least squares algorithm the superior performance of the algorithm is verified via simulation studies

a study of parts based object class detection using complete graphs object detection is one of the key components in modern computer vision systems while the detection of a specific rigid object under changing viewpoints was considered hard just a few years ago current research strives to detect and recognize classes of non rigid articulated objects hampered by the omnipresent confusing information due to clutter and occlusion the focus has shifted from holistic approaches for object detection to representations of individual object parts linked by structural information along with richer contextual descriptions of object configurations along this line of research we present a practicable and expandable probabilistic framework for parts based object class representation enabling the detection of rigid and articulated object classes in arbitrary views we investigate learning of this representation from labelled training images and infer globally optimal solutions to the contextual map detection problem using a search with a novel lower bound as admissible heuristic an assessment of the inference performance of belief propagation and tree reweighted belief propagation is obtained as a by product the generality of our approach is demonstrated on four different datasets utilizing domain dependent information cues

bayesian density regression summary the paper considers bayesian methods for density regression allowing a random probability distribution to change flexibly with multiple predictors the conditional response distribution is expressed as a non parametric mixture of regression models with the mixture distribution changing with predictors a class of weighted mixture of dirichlet process priors is proposed for the uncountable collection of mixture distributions it is shown that this specification results in a generalized polya urn scheme which incorporates weights that are dependent on the distance between subjects predictor values to allow local dependence in the mixture distributions we propose a kernel based weighting scheme a gibbs sampling algorithm is developed for posterior computation the methods are illustrated by using simulated data examples and an epidemiologic application

brief a directional forgetting algorithm based on the decomposition of the information matrix a novel algorithm for directional forgetting is proposed based on a matrix decomposition method which is developed in this paper this algorithm performs exponential forgetting according to the direction of the data vector thus preventing the problem known as estimator windup which is a characteristic of the standard exponential forgetting algorithm

density estimation by mixture models with smoothing priors in the statistical approach for self organizing maps soms learning is regarded as an estimation algorithm for a gaussian mixture model with a gaussian smoothing prior on the centroid parameters the values of the hyperparameters and the topological structure are selected on the basis of a statistical principle however since the component selection probabilities are fixed to a common value the centroids concentrate on areas with high data density this deforms a coordinate system on an extracted manifold and makes smoothness evaluation for the manifold inaccurate in this article we study an extended som model whose component selection probabilities are variable to stabilize the estimation a smoothing prior on the component selection probabilities is introduced an estimation algorithm for the parameters and the hyperparameters based on empirical bayesian inference is obtained the performance of density estimation by the new model and the som model is compared via simulation experiments

shape matching and object recognition using shape contexts we present a novel approach to measuring similarity between shapes and exploit it for object recognition in our framework the measurement of similarity is preceded by 1 solving for correspondences between points on the two shapes 2 using the correspondences to estimate an aligning transform in order to solve the correspondence problem we attach a descriptor the shape context to each point the shape context at a reference point captures the distribution of the remaining points relative to it thus offering a globally discriminative characterization corresponding points on two similar shapes will have similar shape contexts enabling us to solve for correspondences as an optimal assignment problem given the point correspondences we estimate the transformation that best aligns the two shapes regularized thin plate splines provide a flexible class of transformation maps for this purpose the dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points together with a term measuring the magnitude of the aligning transform we treat recognition in a nearest neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image results are presented for silhouettes trademarks handwritten digits and the coil data set

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

the evidence framework applied to classification networks three bayesian ideas are presented for supervised adaptive classifiers first it is argued that the output of a classifier should be obtained by marginalizing over the posterior distribution of the parameters a simple approximation to this integral is proposed and demonstrated this involves a moderation of the most probable classifier s outputs and yields improved performance second it is demonstrated that the bayesian framework for model comparison described for regression models in mackay 1992a b can also be applied to classification problems this framework successfully chooses the magnitude of weight decay terms and ranks solutions found using different numbers of hidden units third an information based data selection criterion is derived and demonstrated within this framework

a tree structured adaptive network for function approximation in high dimensional spaces nonlinear function approximation is often solved by finding a set of coefficients for a finite number of fixed nonlinear basis functions however if the input data are drawn from a high dimensional space the number of required basis functions grows exponentially with dimension leading many to suggest the use of adaptive nonlinear basis functions whose parameters can be determined by iterative methods the author proposes a technique based on the idea that for most of the data only a few dimensions of the input may be necessary to compute the desired output function additional input dimensions are incorporated only where needed the learning procedure grows a tree whose structure depends upon the input data and the function to be approximated this technique has a fast learning algorithm with no local minima once the network shape is fixed and it can be used to reduce the number of required measurements in situations where there is a cost associated with sensing three examples are given controlling the dynamics of a simulated planar two joint robot arm predicting the dynamics of the chaotic mackey glass equation and predicting pixel values in real images from pixel values above and to the left

recovering 3d human pose from monocular images we describe a learning based method for recovering 3d human body pose from single images and monocular image sequences our approach requires neither an explicit body model nor prior labeling of body parts in the image instead it recovers pose by direct nonlinear regression against shape descriptor vectors extracted automatically from image silhouettes for robustness against local silhouette segmentation errors silhouette shape is encoded by histogram of shape contexts descriptors we evaluate several different regression methods ridge regression relevance vector machine rvm regression and support vector machine svm regression over both linear and kernel bases the rvms provide much sparser regressors without compromising performance and kernel bases give a small but worthwhile improvement in performance the loss of depth and limb labeling information often makes the recovery of 3d pose from single silhouettes ambiguous to handle this the method is embedded in a novel regressive tracking framework using dynamics from the previous state estimate together with a learned regression value to disambiguate the pose we show that the resulting system tracks long sequences stably for realism and good generalization over a wide range of viewpoints we train the regressors on images resynthesized from real human motion capture data the method is demonstrated for several representations of full body pose both quantitatively on independent but similar test data and qualitatively on real image sequences mean angular errors of 4 6 spl deg are obtained for a variety of walking motions

global stability proofs for continuous time indirect adaptive control schemes this note presents a general stability proof for continuous time adaptive control with very general assumptions on the identifier and controller we show that if the exogenous input signal is rich enough then both the identifier and the controller converge to their true values applications of the proof to pole placement design and design based on the factorization approach are discussed

inferring decision trees using the minimum description length principle abstract we explore the use of rissanen s minimum description length principle for the construction of decision trees empirical results comparing this approach to other methods are given

asymptotic convergence rate of the em algorithm for gaussian mixtures it is well known that the convergence rate of the expectation maximization em algorithm can be faster than those of convention first order iterative algorithms when the overlap in the given mixture is small but this argument has not been mathematically proved yet this article studies this problem asymptotically in the setting of gaussian mixtures under the theoretical framework of xu and jordan 1996 it has been proved that the asymptotic convergence rate of the em algorithm for gaussian mixtures locally around the true solution is o e0 5 e where e 0 is an arbitrarily small number o x means that it is a higher order infinitesimal as x 0 and e is a measure of the average overlap of gaussians in the mixture in other words the large sample local convergence rate for the em algorithm tends to be asymptotically superlinear when e tends to zero

globally optimal estimation of nonrigid image distortion image alignment in the presence of non rigid distortions is a challenging task typically this involves estimating the parameters of a dense deformation field that warps a distorted image back to its undistorted template generative approaches based on parameter optimization such as lucas kanade can get trapped within local minima on the other hand discriminative approaches like nearest neighbor require a large number of training samples that grows exponentially with respect to the dimension of the parameter space and polynomially with the desired accuracy 1 in this work we develop a novel data driven iterative algorithm that combines the best of both generative and discriminative approaches for this we introduce the notion of a pull back operation that enables us to predict the parameters of the test image using training samples that are not in its neighborhood not close in the parameter space we prove that our algorithm converges to the global optimum using a significantly lower number of training samples that grows only logarithmically with the desired accuracy we analyze the behavior of our algorithm extensively using synthetic data and demonstrate successful results on experiments with complex deformations due to water and clothing