feature adaptive co segmentation by complexity awareness in this paper we propose a novel feature adaptive co segmentation method that can learn adaptive features of different image groups for accurate common objects segmentation we also propose image complexity awareness for adaptive feature learning in the proposed method the original images are first ranked according to the image complexities that are measured by superpixel changing cue and object detection cue then the unsupervised segments of the simple images are used to learn the adaptive features which are achieved using an expectation minimization algorithm combining l 1 regularized least squares optimization with the consideration of the confidence of the simple image segmentation accuracies and the fitness of the learned model the error rate of the final co segmentation is tested by the experiments on different image groups and verified to be lower than the existing state of the art co segmentation methods

nonmonotone enhanced proximal dc algorithms for a class of structured nonsmooth dc programming in this paper we consider a class of structured nonsmooth difference of convex dc minimization in which the first convex component is the sum of a smooth and a nonsmooth function while the second

segmentation of stochastic images with a stochastic random walker method we present an extension of the random walker segmentation to images with uncertain gray values such gray value uncertainty may result from noise or other imaging artifacts or more general from measurement errors in the image acquisition process the purpose is to quantify the influence of the gray value uncertainty onto the result when using random walker segmentation in random walker segmentation a weighted graph is built from the image where the edge weights depend on the image gradient between the pixels for given seed regions the probability is evaluated for a random walk on this graph starting at a pixel to end in one of the seed regions here we extend this method to images with uncertain gray values to this end we consider the pixel values to be random variables rvs thus introducing the notion of stochastic images we end up with stochastic weights for the graph in random walker segmentation and a stochastic partial differential equation pde that has to be solved we discretize the rvs and the stochastic pde by the method of generalized polynomial chaos combining the recent developments in numerical methods for the discretization of stochastic pdes and an interactive segmentation algorithm the resulting algorithm allows for the detection of regions where the segmentation result is highly influenced by the uncertain pixel values thus it gives a reliability estimate for the resulting segmentation and it furthermore allows determining the probability density function of the segmented object volume

iterative scaling and coordinate descent methods for maximum entropy models maximum entropy maxent is useful in natural language processing and many other areas iterative scaling is methods are one of the most popular approaches to solve maxent with many variants of is methods it is difficult to understand them and see the differences in this paper we create a general and unified framework for iterative scaling methods this framework also connects iterative scaling and coordinate descent methods we prove general convergence results for is methods and analyze their computational complexity based on the proposed framework we extend a coordinate descent method for linear svm to maxent results show that it is faster than existing iterative scaling methods

asymptotics for euclidean minimal spanning trees on random points asymptotic results for the euclidean minimal spanning tree onn random vertices inrd can be obtained from consideration of a limiting infinite forest whose vertices form a poisson process in allrd in particular we prove a conjecture of robert bland the sum of thed th powers of the edge lengths of the minimal spanning tree of a random sample ofn points from the uniform distribution in the unit cube ofrd tends to a constant asn

exponentiated gradient algorithms for conditional random fields and max margin markov networks log linear and maximum margin models are two commonly used methods in supervised machine learning and are frequently used in structured prediction problems efficient learning of parameters in these models is therefore an important problem and becomes a key factor when learning from very large data sets this paper describes exponentiated gradient eg algorithms for training such models where eg updates are applied to the convex dual of either the log linear or max margin objective function the dual in both the log linear and max margin cases corresponds to minimizing a convex function with simplex constraints we study both batch and online variants of the algorithm and provide rates of convergence for both cases in the max margin case o 1 e eg updates are required to reach a given accuracy e in the dual in contrast for log linear models only o log 1 e updates are required for both the max margin and log linear cases our bounds suggest that the online eg algorithm requires a factor of n less computation to reach a desired accuracy than the batch eg algorithm where n is the number of training examples our experiments confirm that the online algorithms are much faster than the batch algorithms in practice we describe how the eg updates factor in a convenient way for structured prediction problems allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing we perform extensive evaluation of the algorithms comparing them to l bfgs and stochastic gradient descent for log linear models and to svm struct for max margin models the algorithms are applied to a multi class problem as well as to a more complex large scale parsing task in all these settings the eg algorithms presented here outperform the other methods

manifold constraint transfer for visual structure driven optimization abstract in this paper we leverage the manifold structure of visual data in order to improve performance in general optimization problems subject to linear constraints as the main theoretical result we show that manifold constraints can be transferred from the data to the optimized variables if these are linearly correlated we also show that the resulting optimization problem can be solved with an efficient alternating direction method of multipliers that can consistently integrate the manifold constraints during the optimization process this leads to a simplification of the approach which instead of directly optimizing on the manifold we can iteratively recast the problem as the projection over the manifold via an embedding method the proposed method is extremely versatile since it can be applied to different problems including kernel ridge regression krr and sparse coding which have numerous applications in machine learning and computer vision in particular we apply the methods to different problems such as tracking object recognition and categorization showing a consistent increase of performance with respect to the state of the art

non linear frequency response characteristics of an impact pair abstract the method of harmonic balance is used to develop approximate analytical solutions to the equations of motion of a single impact pair including a general discussion of the existence and stability of these solutions non linear frequency response characteristics are obtained for a single frequency excitation and primary resonance no subharmonic or superharmonic response and the results are validated using an analog computer simulation it has been found that the use of digital simulation numerical integration may lead to inaccurate results over certain frequency ranges it is also shown that although the shapes of the force displacement relationship for clearance non linearity and the classic cubic non linearity are similar the resulting non linear frequency response curves can be quite different various issues related to the clearance non linearity problem are discussed

consistency of spectral clustering consistency is a key property of all statistical procedures analyzing randomly sampled data surprisingly despite decades of work little is known about consistency of most clustering algorithms in this paper we investigate consistency of the popular family of spectral clustering algorithms which clusters the data with the help of eigenvectors of graph laplacian matrices we develop new methods to establish that for increasing sample size those eigenvectors converge to the eigenvectors of certain limit operators as a result we can prove that one of the two major classes of spectral clustering normalized clustering converges under very general conditions while the other unnormalized clustering is only consistent under strong additional assumptions which are not always satisfied in real data we conclude that our analysis provides strong evidence for the superiority of normalized spectral clustering

active contours without edges we propose a new model for active contours to detect objects in a given image based on techniques of curve evolution mumford shah 1989 functional for segmentation and level sets our model can detect objects whose boundaries are not necessarily defined by the gradient we minimize an energy which can be seen as a particular case of the minimal partition problem in the level set formulation the problem becomes a mean curvature flow like evolving the active contour which will stop on the desired boundary however the stopping term does not depend on the gradient of the image as in the classical active contour models but is instead related to a particular segmentation of the image we give a numerical algorithm using finite differences finally we present various experimental results and in particular some examples for which the classical snakes methods based on the gradient are not applicable also the initial curve can be anywhere in the image and interior contours are automatically detected

nested support vector machines one class and cost sensitive support vector machines svms are state of the art machine learning methods for estimating density level sets and solving weighted classification problems respectively however the solutions of these svms do not necessarily produce set estimates that are nested as the parameters controlling the density level or cost asymmetry are continuously varied such nesting not only reflects the true sets being estimated but is also desirable for applications requiring the simultaneous estimation of multiple sets including clustering anomaly detection and ranking we propose new quadratic programs whose solutions give rise to nested versions of one class and cost sensitive svms furthermore like conventional svms the solution paths in our construction are piecewise linear in the control parameters although here the number of breakpoints is directly controlled by the user we also describe decomposition algorithms to solve the quadratic programs these methods are compared to conventional non nested svms on synthetic and benchmark data sets and are shown to exhibit more stable rankings and decreased sensitivity to parameter settings

non linear dynamics of gear pair systems with periodic stiffness and backlash the present work investigates dynamics of a gear pair system involving backlash and time dependent mesh stiffness in addition the system is under the action of external excitation caused by torsional moments and gear geometry errors first the equation of motion is established in a strongly non linear form then the emphasis is laid on a specific forcing frequency range corresponding to conditions of simultaneous fundamental parametric resonance and principal external resonance for these conditions several types of periodic steady state response are identified and determined by employing suitable methodologies including techniques applicable to piecewise linear systems and to oscillators with time periodic coefficients moreover these methodologies are complemented by appropriate procedures revealing the stability properties of the located periodic solutions in the second part of the work numerical results are presented these results verify the validity and effectiveness of the new analytical methodology and provide information on the gear pair dynamics first series of typical response diagrams are obtained illustrating the effect of the mesh stiffness variation the damping and the forcing parameters on the gear pair periodic response these response diagrams are accompanied by results obtained with direct integration of the equation of motion in this way it is demonstrated that for some parameter combinations the dynamical system examined can exhibit more complicated and irregular response including crises and intermittent chaos

an interior point method for large scale l 1 regularized logistic regression logistic regression with l1 regularization has been proposed as a promising method for feature selection in classification problems in this paper we describe an efficient interior point method for solving large scale l1 regularized logistic regression problems small problems with up to a thousand or so features and examples can be solved in seconds on a pc medium sized problems with tens of thousands of features and examples can be solved in tens of seconds assuming some sparsity in the data a variation on the basic method that uses a preconditioned conjugate gradient method to compute the search step can solve very large problems with a million features and examples e g the 20 newsgroups data set in a few minutes on a pc using warm start techniques a good approximation of the entire regularization path can be computed much more efficiently than by solving a family of problems independently

testing over representation of observations in subsets of a dea technology abstract this paper proposes a test for whether data are over represented in a given production zone i e a subset of a production possibility set which has been estimated using the non parametric data envelopment analysis dea approach a binomial test is used that relates the number of observations inside such a zone to a discrete probability weighted relative volume of that zone a monte carlo simulation illustrates the performance of the proposed test statistic and provides good estimation of both facet probabilities and the assumed common inefficiency distribution in a three dimensional input space potential applications include tests for whether benchmark units dominate more or less observations than expected

nearest neighbor clustering a baseline method for consistent clustering with arbitrary objective functions clustering is often formulated as a discrete optimization problem the objective is to find among all partitions of the data set the best one according to some quality measure however in the statistical setting where we assume that the finite data set has been sampled from some underlying space the goal is not to find the best partition of the given sample but to approximate the true partition of the underlying space we argue that the discrete optimization approach usually does not achieve this goal and instead can lead to inconsistency we construct examples which provably have this behavior as in the case of supervised learning the cure is to restrict the size of the function classes under consideration for appropriate small function classes we can prove very general consistency theorems for clustering optimization schemes as one particular algorithm for clustering with a restricted function space we introduce nearest neighbor clustering similar to the k nearest neighbor classifier in supervised learning this algorithm can be seen as a general baseline algorithm to minimize arbitrary clustering objective functions we prove that it is statistically consistent for all commonly used clustering objective functions

a pseudo heuristic parameter selection rule for l1 regularized minimization problems abstract this paper considers the regularization parameter determination of l 1 regularized minimization problem we solve the l 1 regularized problem using iterative reweighted least squares irls which involves solving a linear system whose coefficient matrix has the form m 1 n 0 1 the aim of this paper is to find an efficient and computationally inexpensive algorithm to both choose the regularization parameter and solve the l 1 regularized problem in order to achieve this we propose an irls algorithm with adaptive regularization parameter selection based on a heuristic parameter determination rule de boor s parameter selection criterion compared with some of the state of the art algorithms and parameter selection rules the numerical experiments show the efficiency and robustness of the proposed method

random walks for image segmentation a novel method is proposed for performing multilabel interactive image segmentation given a small number of pixels with user defined or predefined labels one can analytically and quickly determine the probability that a random walker starting at each unlabeled pixel will first reach one of the prelabeled pixels by assigning each pixel to the label for which the greatest probability is calculated a high quality image segmentation may be obtained theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits this algorithm is formulated in discrete space i e on a graph using combinatorial analogues of standard operators and principles from continuous potential theory allowing it to be applied in arbitrary dimension on arbitrary graphs

a novel approach to fruc using discriminant saliency and frame segmentation motion compensated frame interpolation mcfi is a technique used extensively for increasing the temporal frequency of a video sequence in order to obtain a high quality interpolation the motion field between frames must be well estimated however many current techniques for determining the motion are prone to errors in occlusion regions as well as regions with repetitive structure we propose an algorithm for improving both the objective and subjective quality of mcfi by refining the motion vector field we first utilize a discriminant saliency classifier to determine which regions of the motion field are most important to a human observer these regions are refined using a multistage motion vector refinement mvr which promotes motion vector candidates based upon their likelihood given a local neighborhood for regions which fall below the saliency threshold a frame segmentation is used to locate regions of homogeneous color and texture via normalized cuts motion vectors are promoted such that each homogeneous region has a consistent motion experimental results demonstrate an improvement over previous frame rate up conversion fruc methods in both objective and subjective picture quality

shape classification using the inner distance part structure and articulation are of fundamental importance in computer and human vision we propose using the inner distance to build shape descriptors that are robust to articulation and capture part structure the inner distance is defined as the length of the shortest path between landmark points within the shape silhouette we show that it is articulation insensitive and more effective at capturing part structures than the euclidean distance this suggests that the inner distance can be used as a replacement for the euclidean distance to build more accurate descriptors for complex shapes especially for those with articulated parts in addition texture information along the shortest path can be used to further improve shape classification with this idea we propose three approaches to using the inner distance the first method combines the inner distance and multidimensional scaling mds to build articulation invariant signatures for articulated shapes the second method uses the inner distance to build a new shape descriptor based on shape contexts the third one extends the second one by considering the texture information along shortest paths the proposed approaches have been tested on a variety of shape databases including an articulated shape data set mpeg7 ce shape 1 kimia silhouettes the eth 80 data set two leaf data sets and a human motion silhouette data set in all the experiments our methods demonstrate effective performance compared with other algorithms

period one motions of a mechanical oscillator with periodically time varying piecewise nonlinear stiffness the dynamic behavior of a piecewise nonlinear mechanical oscillator with parametric and external excitations is investigated the viscously damped single degree of freedom oscillator is subjected to a periodically time varying piecewise nonlinear restoring function defined by a clearance surrounded by continuously nonlinear quadratic and cubic regions typical applications represented by this oscillator are highlighted a multi term harmonic balance formulation is used in conjunction with discrete fourier transforms and a parametric continuation scheme to determine steady state period 1 motions of the system due to both parametric and external excitations the accuracy of the analytical solutions is demonstrated by comparing them to direct numerical integration solutions floquet theory is applied to determine the stability of the steady state harmonic balance solutions at the end detailed parametric studies are presented to quantify the combined influence of clearance quadratic and cubic nonlinearities within reasonable ranges of all other system parameters a comparison between time varying and time invariant systems is also provided to demonstrate the influence of the parametric and external excitations on a piecewise nonlinear system

a hybrid clustering method for identifying high density clusters abstract a hybrid method for clustering multivariate observations is proposed which combines elements of the k means and the single linkage clustering techniques one purpose of the proposed method is to discover the high density clusters given a random sample of size n from some underlying population a high density cluster at level c in a population with density f is defined as a maximal connected set of points x with f x c this clustering procedure is practicable for very large numbers of observations and is shown to be consistent under certain regularity conditions in one dimension

the mapping dynamics of periodic motions for a three piecewise linear system under a periodic excitation in this paper the mapping dynamics of periodic motions for a three piecewise linear system under a periodic excitation is developed and the mapping structures for specified periodic motions are constructed based on the mapping structures an analytical prediction of all possible stable and unstable periodic motions is given the symmetry for the stable asymmetrical periodic motions of such a system is observed however the unstable periodic motions do not have such symmetry the methodology presented in this paper is applicable to other non smooth systems such as friction induced vibration impact oscillator and power control systems in addition the mapping dynamics provides a useful and efficient tool for the co existence of periodic motions and or chaotic motions in nonlinear dynamical systems

graph cuts via ell_1 norm minimization graph cuts have become an increasingly important tool for solving a number of energy minimization problems in computer vision and other fields in this paper the graph cut problem is reformulated as an unconstrained l1 norm minimization that can be solved effectively using interior point methods this reformulation exposes connections between graph cuts and other related continuous optimization problems eventually the problem is reduced to solving a sequence of sparse linear systems involving the laplacian of the underlying graph the proposed procedure exploits the structure of these linear systems in a manner that is easily amenable to parallel implementations experimental results obtained by applying the procedure to graphs derived from image processing problems are provided

metric learning for image alignment image alignment has been a long standing problem in computer vision parameterized appearance models pams such as the lucas kanade method eigentracking and active appearance models are commonly used to align images with respect to a template or to a previously learned model while pams have numerous advantages relative to alternate approaches they have at least two drawbacks first they are especially prone to local minima in the registration process second often few if any of the local minima of the cost function correspond to acceptable solutions to overcome these problems this paper proposes a method to learn a metric for pams that explicitly optimizes that local minima occur at and only at the places corresponding to the correct fitting parameters to the best of our knowledge this is the first paper to address the problem of learning a metric to explicitly model local properties of the pams error surface synthetic and real examples show improvement in alignment performance in comparison with traditional approaches in addition we show how the proposed criteria for a good metric can be used to select good features to track

nonlinear analysis of a geared rotor system supported by fluid film journal bearings abstract this paper presents a novel approach for modeling and analyzing a geared rotor bearing system including nonlinear forces in the gear set and the supporting fluid film journal bearings the rotordynamics system model has five degrees of freedom that define the transverse displacements of the shaft gear centerlines and the relative displacement of the gear tooth contact point the journal bearing nonlinear forces are obtained via a solution of reynolds equation for lubricant film pressure utilizing the finite element method co existing steady state autonomous and non autonomous responses are obtained in an accurate and computationally efficient manner utilizing the multiple shooting and continuation algorithms this yields the full manifolds of the multiple bifurcation system chaos is identified with maximum lyapunov exponents frequency spectra poincare attractors etc the results reveal a dependence of the gear set contact conditions and system nonlinear response characteristics i e jump co existing responses subharmonic resonances and chaos on the choice of journal bearing parameters the results also show that hopf bifurcations which occur along with oil whirl in a journal bearing system can be attenuated by increasing the gear torque

characterization stability and convergence of hierarchical clustering methods we study hierarchical clustering schemes under an axiomatic view we show that within this framework one can prove a theorem analogous to one of kleinberg 2002 in which one obtains an existence and uniqueness theorem instead of a non existence result we explore further properties of this unique scheme stability and convergence are established we represent dendrograms as ultrametric spaces and use tools from metric geometry namely the gromov hausdorff distance to quantify the degree to which perturbations in the input metric space affect the result of hierarchical methods

complementary saliency driven co segmentation with region searching and hierarchical constraint foreground histogram consistency is a commonly used global constraint for co segmentation problems however scale target posture as well as the viewpoint changes usually make it hard to guarantee the absolute consistency of histogram in this paper we formulate a unified framework by incorporating the local region searching strategy and hierarchical constraint with the global scale invariant framework additionally complementary saliency is adopted to reduce the unreliable prediction brought by single saliency estimation method such automatic foreground inferring strategy also shows good compatibility and interaction with the new unified co segmentation framework the comparison experiments conducted on the public datasets demonstrate the good performance of the proposed method

the dip test of unimodality the dip test measures multimodality in a sample by the maximum difference over all sample points between the empirical distribution function and the unimodal distribution function that minimizes that maximum difference the uniform distribution is the asymptotically least favorable unimodal distribution and the distribution of the test statistic is determined asymptotically and empirically when sampling from the uniform

active appearance models revisited active appearance models aams and the closely related concepts of morphable models and active blobs are generative models of a certain visual phenomenon although linear in both shape and appearance overall aams are nonlinear parametric models in terms of the pixel intensities fitting an aam to an image consists of minimising the error between the input image and the closest model instances i e solving a nonlinear optimisation problem we propose an efficient fitting algorithm for aams based on the inverse compositional image alignment algorithm we show that the effects of appearance variation during fitting can be precomputed projected out using this algorithm and how it can be extended to include a global shape normalising warp typically a 2d similarity transformation we evaluate our algorithm to determine which of its novel aspects improve aam fitting performance

rationalizing efficient compositional image alignment we study the issue of computational efficiency for gauss newton gn non linear least squares optimization in the context of image alignment we introduce the constant jacobian gauss newton cjgn optimization a gn scheme with constant jacobian and hessian matrices and the equivalence and independence conditions as the necessary requirements that any function of residuals must satisfy to be optimized with this efficient approach we prove that the inverse compositional ic image alignment algorithm is an instance of a cjgn scheme and formally derive the compositional and extended brightness constancy assumptions as the necessary requirements that must be satisfied by any image alignment problem so it can be solved with an efficient compositional scheme moreover in contradiction with previous results we also prove that the forward and inverse compositional algorithms are not equivalent they are equivalent however when the extended brightness constancy assumption is satisfied to analyze the impact of the satisfaction of these requirements we introduce a new image alignment evaluation framework and the concepts of short and wide baseline jacobian in wide baseline jacobian problems the optimization will diverge if the requirements are not satisfied however with a good initialization a short baseline jacobian problem may converge even if the requirements are not satisfied

generalized parametric resonance in electrostatically actuated microelectromechanical oscillators this paper investigates the dynamic response of a class of electrostatically driven microelectromechanical mem oscillators the particular systems of interest are those which feature parametric excitation that arises from forces produced by fluctuating voltages applied across comb drives these systems are known to exhibit a wide range of behaviors some of which have escaped explanation or prediction in this paper we examine a general governing equation of motion for these systems and use it to provide a complete description of the dynamic response and its dependence on the system parameters the defining feature of this equation is that both the linear and cubic terms feature parametric excitation which in comparison to the case of purely linear parametric excitation e g the mathieu equation significantly complicates the system s dynamics one consequence is that an effective nonlinearity for the overall system cannot be defined instead the system features separate effective nonlinearities for each branch of its nontrivial response as such it can exhibit not only hardening and softening nonlinearities but also mixed nonlinearities wherein the response branches in the system s frequency response bend toward or away from one another near resonance this paper includes some brief background information on the equation of motion under consideration an outline of the analytical techniques used to reach the aforementioned results stability results for the responses in question a numerical example explored using simulation of a mem oscillator which features this nonlinear behavior and preliminary experimental results taken from an actual mem device which show evidence of the analytically predicted behavior practical issues pertaining to the design of parametrically excited mem devices are also considered

procedures for the identification of multiple outliers in linear models abstract we consider the problem of identifying and testing multiple outliers in linear models the available outlier identification methods often do not succeed in detecting multiple outliers because they are affected by the observations they are supposed to identify we introduce two test procedures for the detection of multiple outliers that appear to be less sensitive to this problem both procedures attempt to separate the data into a set of clean data points and a set of points that contain the potential outliers the potential outliers are then tested to see how extreme they are relative to the clean subset using an appropriately scaled version of the prediction error the procedures are illustrated and compared to various existing methods using several data sets known to contain multiple outliers also the performances of both procedures are investigated by a monte carlo study the data sets and the monte carlo indicate that both procedures are effective in the detection of multiple outliers

iterative reweighted minimization methods for l_p lp regularized unconstrained nonlinear programming in this paper we study general l p l p regularized unconstrained minimization problems in particular we derive lower bounds for nonzero entries of the first and second order stationary points and hence also of local minimizers of the l p l p minimization problems we extend some existing iterative reweighted l 1 l 1 mathrm irl 1 irl 1 and l 2 l 2 mathrm irl 2 irl 2 minimization methods to solve these problems and propose new variants for them in which each subproblem has a closed form solution also we provide a unified convergence analysis for these methods in addition we propose a novel lipschitz continuous epsilon approximation to vert x vert p p x p p using this result we develop new mathrm irl 1 irl 1 methods for the l p l p minimization problems and show that any accumulation point of the sequence generated by these methods is a first order stationary point provided that the approximation parameter epsilon is below a computable threshold value this is a remarkable result since all existing iterative reweighted minimization methods require that epsilon be dynamically updated and approach zero our computational results demonstrate that the new mathrm irl 1 irl 1 method and the new variants generally outperform the existing mathrm irl 1 irl 1 methods chen and zhou in 2012 foucart and lai in appl comput harmon anal 26 395 407 2009

on the complexity of logistic regression models we investigate the complexity of logistic regression models which is defined by counting the number of indistinguishable distributions that the model can represent balasubramanian 1997 we find

active appearance models we describe a new method of matching statistical models of appearance to images a set of model parameters control modes of shape and gray level variation learned from a training set we construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors

trust region newton method for logistic regression large scale logistic regression arises in many applications such as document classification and natural language processing in this paper we apply a trust region newton method to maximize the log likelihood of the logistic regression model the proposed method uses only approximate newton steps in the beginning but achieves fast convergence in the end experiments show that it is faster than the commonly used quasi newton approach for logistic regression we also extend the proposed method to large scale l2 loss linear support vector machines svm

bayesian constrained local models revisited this paper presents a novel bayesian formulation for aligning faces in unseen images our approach revisits the constrained local models clm formulation where an ensemble of local feature detectors are constrained to lie within the subspace spanned by a point distribution model pdm fitting such a model to an image typically involves two main steps a local search using a detector obtaining response maps for each landmark likelihood term and a global optimization that finds the pdm parameters that jointly maximize all the detections at once the so called global optimization can be posed as a bayesian inference problem where the posterior distribution of the shape and pose parameters can be inferred in a maximum a posteriori map sense this work introduces an extended bayesian global optimization strategy that includes two novel additions 1 to perform second order updates of the pdm parameters accounting for their covariance and 2 to model the underlying dynamics of the shape variations encoded in the prior term by using recursive bayesian estimation extensive evaluations were performed against state of the art methods on several standard datasets imm bioid xm2vts lfw and fgnet talking face results show that the proposed approach significantly increases the fitting performance

a co saliency model of image pairs in this paper we introduce a method to detect co saliency from an image pair that may have some objects in common the co saliency is modeled as a linear combination of the single image saliency map sism and the multi image saliency map mism the first term is designed to describe the local attention which is computed by using three saliency detection techniques available in literature to compute the mism a co multilayer graph is constructed by dividing the image pair into a spatial pyramid representation each node in the graph is described by two types of visual descriptors which are extracted from a representation of some aspects of local appearance e g color and texture properties in order to evaluate the similarity between two nodes we employ a normalized single pair simrank algorithm to compute the similarity score experimental evaluation on a number of image pairs demonstrates the good performance of the proposed method on the co saliency detection task

high dimensional ising model selection using l1 regularized logistic regression we consider the problem of estimating the graph associated with a binary ising markov random field we describe a method based on l1 regularized logistic regression in which the neighborhood of any given node is estimated by performing logistic regression subject to an l1 constraint the method is analyzed under high dimensional scaling in which both the number of nodes p and maximum neighborhood size d are allowed to grow as a function of the number of observations n our main results provide sufficient conditions on the triple n p d and the model parameters for the method to succeed in consistently estimating the neighborhood of every node in the graph simultaneously with coherence conditions imposed on the population fisher information matrix we prove that consistent neighborhood selection can be obtained for sample sizes n d3log p with exponentially decaying error when these same conditions are imposed directly on the sample matrices we show that a reduced sample size of n d2log p suffices for the method to estimate neighborhoods consistently although this paper focuses on the binary graphical models we indicate how a generalization of the method of the paper would apply to general discrete markov random fields

a kernel classification framework for metric learning learning a distance metric from the given training samples plays a crucial role in many machine learning tasks and various models and optimization algorithms have been proposed in the past decade in this paper we generalize several state of the art metric learning methods such as large margin nearest neighbor lmnn and information theoretic metric learning itml into a kernel classification framework first doublets and triplets are constructed from the training samples and a family of degree 2 polynomial kernel functions is proposed for pairs of doublets or triplets then a kernel classification framework is established to generalize many popular metric learning methods such as lmnn and itml the proposed framework can also suggest new metric learning methods which can be efficiently implemented interestingly using the standard support vector machine svm solvers two novel metric learning methods namely doublet svm and triplet svm are then developed under the proposed framework experimental results show that doublet svm and triplet svm achieve competitive classification accuracies with state of the art metric learning methods but with significantly less training time

a statistical model of cluster stability in the current paper we present a method for assessing cluster stability this method combined with a clustering algorithm yields an estimate of the data partition namely the number of clusters we adopt the cluster stability standpoint where clusters are imagined as islands of high density in a sea of low density explicitly a cluster is associated with its high density core our approach offers to evaluate the goodness of a cluster by the similarity amongst the entire cluster and its core we propose to measure this resemblance by two sample tests or by probability distances between appropriate probability distributions the distances are calculated on clustered samples drawn from the source population according to two different distributions the first law is the underlying set distribution the second law is constructed so that it represents the clusters cores here a variant of the k nearest neighbor density estimation is applied so that items belonging to cores have a much higher chance to be selected as the sample distribution is unknown a distribution free two sample test is required to examine the mentioned correspondence for constructing such a test we use distance functions built on negative definite kernels in practice outliers in the samples and limitations of the clustering algorithm heavily contribute to the noise level as a result of this shortcoming the distance values have to be determined for many pairs of samples and therefore an empirical distance s distribution is obtained the distribution is dependent on the examined number of clusters to prevent this property for biasing the results we normalize the distances it is conjectured that the true number of clusters yields the most concentrated normalized distribution to measure the concentration we use the sample mean and the sample 25th percentile the paper exhibits the good performance of the proposed method on synthetic and real world data

alternating linearization for structured regularization problems we adapt the alternating linearization method for proximal decomposition to structured regularization problems in particular to the generalized lasso problems the method is related to two well known operator splitting methods the douglas rachford and the peaceman rachford method but it has descent properties with respect to the objective function this is achieved by employing a special update test which decides whether it is beneficial to make a peaceman rachford step any of the two possible douglas rachford steps or none the convergence mechanism of the method is related to that of bundle methods of nonsmooth optimization we also discuss implementation for very large problems with the use of specialized algorithms and sparse data structures finally we present numerical results for several synthetic and real world examples including a three dimensional fused lasso problem which illustrate the scalability efficacy and accuracy of the method

consistency of cheeger and ratio graph cuts this paper establishes the consistency of a family of graph cut based algorithms for clustering of data clouds we consider point clouds obtained as samples of a ground truth measure we investigate approaches to clustering based on minimizing objective functionals defined on proximity graphs of the given sample our focus is on functionals based on graph cuts like the cheeger and ratio cuts we show that minimizers of these cuts converge as the sample size increases to a minimizer of a corresponding continuum cut which partitions the ground truth measure moreover we obtain sharp conditions on how the connectivity radius can be scaled with respect to the number of sample points for the consistency to hold we provide results for two way and for multiway cuts furthermore we provide numerical experiments that illustrate the results and explore the optimality of scaling in dimension two

optimal construction of k nearest neighbor graphs for identifying noisy clusters we study clustering algorithms based on neighborhood graphs on a random sample of data points the question we ask is how such a graph should be constructed in order to obtain optimal clustering results which type of neighborhood graph should one choose mutual k nearest neighbor or symmetric k nearest neighbor what is the optimal parameter k in our setting clusters are defined as connected components of the t level set of the underlying probability distribution clusters are said to be identified in the neighborhood graph if connected components in the graph correspond to the true underlying clusters using techniques from random geometric graph theory we prove bounds on the probability that clusters are identified successfully both in a noise free and in a noisy setting those bounds lead to several conclusions first k has to be chosen surprisingly high rather of the order n than of the order logn to maximize the probability of cluster identification secondly the major difference between the mutual and the symmetric k nearest neighbor graph occurs when one attempts to detect the most significant cluster only

strong rules for discarding predictors in lasso type problems summary we consider rules for discarding predictors in lasso regression and related problems for computational efficiency el ghaoui and his colleagues have proposed safe rules based on univariate inner products between each predictor and the outcome which guarantee that a coefficient will be 0 in the solution vector this provides a reduction in the number of variables that need to be entered into the optimization we propose strong rules that are very simple and yet screen out far more predictors than the safe rules this great practical improvement comes at a price the strong rules are not foolproof and can mistakenly discard active predictors i e predictors that have non zero coefficients in the solution we therefore combine them with simple checks of the karush kuhn tucker conditions to ensure that the exact solution to the convex problem is delivered of course any approximate screening method can be combined with the karush kuhn tucker conditions to ensure the exact solution the strength of the strong rules lies in the fact that in practice they discard a very large number of the inactive predictors and almost never commit mistakes we also derive conditions under which they are foolproof strong rules provide substantial savings in computational time for a variety of statistical optimization problems

unsupervised multi class co segmentation via joint cut over l_ 1 manifold hyper graph of discriminative image regions this paper systematically advocates a robust and efficient unsupervised multi class co segmentation approach by leveraging underlying subspace manifold propagation to exploit the cross image coherency it can combat certain image co segmentation difficulties due to viewpoint change partial occlusion complex background transient illumination and cluttering texture patterns our key idea is to construct a powerful hyper graph joint cut framework which incorporates mid level image regions based intra image feature representation and l 1 manifold graph based inter image coherency exploration for local image region generation we propose a bi harmonic distance distribution difference metric to govern the super pixel clustering in a bottom up way it not only affords drastic data reduction but also gives rise to discriminative and structure meaningful feature representation as for the inter image coherency we leverage multi type features involved l 1 graph to detect the underlying local manifold from cross image regions as a result the implicit supervising information could be encoded into the unsupervised hyper graph joint cut framework we conduct extensive experiments and make comprehensive evaluations with other state of the art methods over various benchmarks including icoseg msrc and oxford flower all the results demonstrate the superiorities of our method in terms of accuracy robustness efficiency and versatility

eigentracking robust matching and tracking of articulated objects using a view based representation this paper describes an approach for tracking rigid and articulated objects using a view based representation the approach builds on and extends work on eigenspace representations robust estimation techniques and parameterized optical flow estimation first we note that the least squares image reconstruction of standard eigenspace techniques has a number of problems and we reformulate the reconstruction problem as one of robust estimation second we define a subspace constancy assumption that allows us to exploit techniques for parameterized optical flow estimation to simultaneously solve for the view of an object and the affine transformation between the eigenspace and the image to account for large affine transformations between the eigenspace and the image we define a multi scale eigenspace representation and a coarse to fine matching strategy finally we use these techniques to track objects over long image sequences in which the objects simultaneously undergo both affine image motions and changes of view in particular we use this eigentracking technique to track and recognize the gestures of a moving hand

multiscale methods for polyhedral regularizations in this paper we present the extension and generalization of the adaptive inverse scale space aiss method proposed for ell 1 regularization in math comp 82 2013 pp 269 299 to arbitrary polyhedral functions we will see that the representation of a convex polyhedral function as a finitely generated function allows us to conclude that the inverse scale space flow the time continuous formulation of the augmented lagrangian method can be solved exactly without discretization we present an algorithm for the computation of such a solution for arbitrary polyhedral functions analyze its convergence and interpret the well known forward scale space flow as the inverse scale space flow on the convex conjugate functional thus including this class of flows in our analysis a surprising result is the equivalence of the scale space or gradient flow with a standard variational problem finally we give examples of the applications for the aiss algorithm for polyhedral functions which illustrate th

fully adaptive density based clustering the clusters of a distribution are often defined by the connected components of a density level set however this definition depends on the user specified level we address this issue by proposing a simple generic algorithm which uses an almost arbitrary level set estimator to estimate the smallest level at which there are more than one connected components in the case where this algorithm is fed with histogram based level set estimates we provide a finite sample analysis which is then used to show that the algorithm consistently estimates both the smallest level and the corresponding connected components we further establish rates of convergence for the two estimation problems and last but not least we present a simple yet adaptive strategy for determining the width parameter of the involved density estimator in a data depending way

weakly supervised part proposal segmentation from multiple images weakly supervised local part segmentation is challenging due to the difficulty of modeling multiple local parts from image level prior in this paper we propose a new weakly supervised local part proposal segmentation method based on the observation that local parts will keep fixed along the object pose variations hence the local part can be segmented by capturing object pose variations based on such observation a new local part proposal segmentation model is proposed three aspects such as shape similarity based cosegmentation shape matching based part detection and segmentation and graph matching based part assignment are considered a part segmentation energy function is first proposed four terms such as mrf based single image segmentation term shape feature based foreground consistency term ncuts based part segmentation term and two order graphs matching based part consistency term are contained then a three sub minimization based energy minimization method is proposed to accomplish approximation solution finally we verify our method based on three image data sets pascal voc 2008 part data set ucb bird data set and cat dog data set and one video data set ucf sports data set the experimental results demonstrate a better segmentation performance compared with the existing object cosegmentation and part proposal generation methods

consistent procedures for cluster tree estimation and pruning for a density f on mathbb r d a high density cluster is any connected component of x f x geq lambda for some lambda 0 the set of all high density clusters forms a hierarchy called the cluster tree of f we present two procedures for estimating the cluster tree given samples from f the first is a robust variant of the single linkage algorithm for hierarchical clustering the second is based on the k nearest neighbor graph of the samples we give finite sample convergence rates for these algorithms which also imply consistency and we derive lower bounds on the sample complexity of cluster tree estimation finally we study a tree pruning procedure that guarantees under milder conditions than usual to remove clusters that are spurious while recovering those that are salient

interacting multiview tracker a robust algorithm is proposed for tracking a target object in dynamic conditions including motion blurs illumination changes pose variations and occlusions to cope with these challenging factors multiple trackers based on different feature representations are integrated within a probabilistic framework each view of the proposed multiview multi channel feature learning algorithm is concerned with one particular feature representation of a target object from which a tracker is developed with different levels of reliability with the multiple trackers the proposed algorithm exploits tracker interaction and selection for robust tracking performance in the tracker interaction a transition probability matrix is used to estimate dependencies between trackers multiple trackers communicate with each other by sharing information of sample distributions the tracker selection process determines the most reliable tracker with the highest probability to account for object appearance changes the transition probability matrix and tracker probability are updated in a recursive bayesian framework by reflecting the tracker reliability measured by a robust tracker likelihood function that learns to account for both transient and stable appearance changes experimental results on benchmark datasets demonstrate that the proposed interacting multiview algorithm performs robustly and favorably against state of the art methods in terms of several quantitative metrics

estimation of nonlinear errors in variables models for computer vision applications in an errors in variables eiv model all the measurements are corrupted by noise the class of eiv models with constraints separable into the product of two nonlinear functions one solely in the variables and one solely in the parameters is general enough to represent most computer vision problems we show that the estimation of such nonlinear eiv models can be reduced to iteratively estimating a linear model having point dependent i e heteroscedastic noise process particular cases of the proposed heteroscedastic errors in variables heiv estimator are related to other techniques described in the vision literature the sampson method renormalization and the fundamental numerical scheme in a wide variety of tasks the heiv estimator exhibits the same or superior performance as these techniques and has a weaker dependence on the quality of the initial solution than the levenberg marquardt method the standard approach toward estimating nonlinear models

consistency of single linkage for high density clusters abstract high density clusters are defined on a population with density f in r dimensions to be the maximal connected sets of form x f x c single linkage clustering is evaluated for consistency in detecting such high density clusters other standard hierarchical techniques such as average and complete linkage are hopelessly inconsistent for these clusters the asymptotic consistency of single linkage closely depends on the percolation problem of broadbent and hammersley if small spheres are removed at random from a solid at which density of spheres will water begin to flow through the solid if there is a single critical density such that no flow takes place below a certain density and flow occurs through a single connected set above that density then single linkage is consistent in separating high density clusters by disjoint single linkage clusters that include a positive fraction of sample points in the respective clusters and pass arbitrarily close to all points in the respective clusters

perturbation analysis of a clearance type nonlinear system abstract this study applies the method of multiple scales to obtain periodic solutions of a two pulley belt system with clearance type nonlinearity the purpose is to explain the published numerical results and clarify how design parameters affect the system dynamics the validity of the perturbation method for such strong nonlinearity is evaluated the closed form frequency response relation is determined at the first order and an implicit expression is obtained for the second order approximation the preload applied to the accessory determines the softening level of the nonlinearity larger preload leads to less disengagement and less softening for a considerable range of practical parameter values the analytical solutions well approximate the numerical results from harmonic balance

analysis of periodically forced nonlinear hill s oscillator with application to a geared system although nonlinear systems subject to combined parametric and external excitations have been examined in some depth very few of these investigations have addressed the influence of a mean time invariant load on the system response in particular the importance of the mean load effect has been highlighted in the study of whirling asymmetric shafts where significant changes in system response and stability occur under the influence of gravity the present paper intends to characterize the specific effect of mean load on the dynamic behavior of a hill s oscillator with a clearance type nonlinearity while also being subjected to a periodic base displacement excitation the parametric continuation technique and method of harmonic balance would be used for this purpose issues discussed would include the coupling between the mean load and the dynamic response amplitude interaction between the parametric excitation effect and the clearance nonlinearity and comparison between time invariant and time varying systems further a geared system has been analyzed as a practical application of the nonlinear hill s equation examined earlier studies on gear dynamics have been reexamined to provide improved correlation with prior experimental data new interpretations for some of the nonlinear phenomena in the experimental data has also been provided based on our study

lucas kanade 20 years on a unifying framework since the lucas kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision applications range from optical flow and tracking to layered motion mosaic construction and face coding numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation we present an overview of image alignment describing most of the algorithms and their extensions in a consistent framework we concentrate on the inverse compositional algorithm an efficient algorithm that we recently proposed we examine which of the extensions to lucas kanade can be used with the inverse compositional algorithm without any significant loss of efficiency and which cannot in this paper part 1 in a series of papers we cover the quantity approximated the warp update rule and the gradient descent approximation in future papers we will cover the choice of the error function how to allow linear appearance variation and how to impose priors on the parameters

manifold identification in dual averaging for regularized stochastic online learning iterative methods that calculate their steps from approximate subgradient directions have proved to be useful for stochastic learning problems over large and streaming data sets when the objective consists of a loss function plus a nonsmooth regularization term the solution often lies on a low dimensional manifold of parameter space along which the regularizer is smooth when an l1 regularizer is used to induce sparsity in the solution for example this manifold is defined by the set of nonzero components of the parameter vector this paper shows that a regularized dual averaging algorithm can identify this manifold with high probability before reaching the solution this observation motivates an algorithmic strategy in which once an iterate is suspected of lying on an optimal or near optimal manifold we switch to a local phase that searches in this manifold thus converging rapidly to a near optimal point computational results are presented to verify the identification property and to illustrate the effectiveness of this approach

nonconvex sparse logistic regression with weakly convex regularization in this paper we propose to fit a sparse logistic regression model by a weakly convex regularized nonconvex optimization problem the idea is based on the finding that a weakly convex function as an approximation of the l 0 pseudo norm is able to better induce sparsity than the commonly used l 1 norm for a class of weakly convex sparsity inducing functions we prove the nonconvexity of the corresponding problem and study its local optimality conditions and the choice of the regularization parameter despite the nonconvexity a method based on proximal gradient descent is used to solve the general weakly convex sparse logistic regression and its convergence behavior is studied theoretically then the general framework is applied to a specific weakly convex function and a local optimality condition and a bound on the logistic loss at a local optimum are provided the solution method is instantiated in this case as an iterative firm shrinkage algorithm and a nesterov acceleration is used with a convergence guarantee its effectiveness is demonstrated in numerical experiments by both randomly generated and real datasets

efficient sparse generalized multiple kernel learning kernel methods have been successfully applied in various applications to succeed in these applications it is crucial to learn a good kernel representation whose objective is to reveal the data similarity precisely in this paper we address the problem of multiple kernel learning mkl searching for the optimal kernel combination weights through maximizing a generalized performance measure most mkl methods employ the norm simplex constraints on the kernel combination weights which therefore involve a sparse but non smooth solution for the kernel weights despite the success of their efficiency they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance alternatively imposing the norm constraint on the kernel weights will keep all the information in the base kernels this leads to non sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information to tackle these problems we propose a generalized mkl gmkl model by introducing an elastic net type constraint on the kernel weights more specifically it is an mkl model with a constraint on a linear combination of the norm and the squared norm on the kernel weights to seek the optimal kernel combination weights therefore previous mkl problems based on the norm or the norm constraints can be regarded as special cases furthermore our gmkl enjoys the favorable sparsity property on the solution and also facilitates the grouping effect moreover the optimization of our gmkl is a convex optimization problem where a local solution is the global optimal solution we further derive a level method to efficiently solve the optimization problem a series of experiments on both synthetic and real world datasets have been conducted to show the effectiveness and efficiency of our gmkl