aggregating local image descriptors into compact codes this paper addresses the problem of large scale image search three constraints have to be taken into account search accuracy efficiency and memory usage we first present and evaluate different ways of aggregating local image descriptors into a vector and show that the fisher kernel achieves better performance than the reference bag of visual words approach for any given vector dimension we then jointly optimize dimensionality reduction and indexing in order to obtain a precise vector comparison as well as a compact representation the evaluation shows that the image representation can be reduced to a few dozen bytes while preserving high accuracy searching a 100 million image data set takes about 250 ms on one processor core

structured multicategory support vector machines with analysis of variance decomposition the support vector machine has been a popular choice of classification method for many applications in machine learning while it often outperforms other methods in terms of classification accuracy the implicit nature of its solution renders the support vector machine less attractive in providing insights into the relationship between covariates and classes use of structured kernels can remedy the drawback borrowing the flexible model building idea of functional analysis of variance decomposition we consider multicategory support vector machines with analysis of variance kernels in this paper an additional penalty is imposed on the sum of weights of functional subspaces which encourages a sparse representation of the solution incorporation of the additional penalty enhances the interpretability of a resulting classifier with often improved accuracy the proposed method is demonstrated through simulation studies and an application to real data

spacetime texture representation and recognition based on a spatiotemporal orientation analysis this paper is concerned with the representation and recognition of the observed dynamics i e excluding purely spatial appearance cues of spacetime texture based on a spatiotemporal orientation analysis the term spacetime texture is taken to refer to patterns in visual spacetime x y t that primarily are characterized by the aggregate dynamic properties of elements or local measurements accumulated over a region of spatiotemporal support rather than in terms of the dynamics of individual constituents examples include image sequences of natural processes that exhibit stochastic dynamics e g fire water and windblown vegetation as well as images of simpler dynamics when analyzed in terms of aggregate region properties e g uniform motion of elements in imagery such as pedestrians and vehicular traffic spacetime texture representation and recognition is important as it provides an early means of capturing the structure of an ensuing image stream in a meaningful fashion toward such ends a novel approach to spacetime texture representation and an associated recognition method are described based on distributions histograms of spacetime orientation structure empirical evaluation on both standard and original image data sets shows the promise of the approach including significant improvement over alternative state of the art approaches in recognizing the same pattern from different viewpoints

sparse sufficient dimension reduction existing sufficient dimension reduction methods suffer from the fact that each dimension reduction component is a linear combination of all the original predictors so that it is difficult to interpret the resulting estimates we propose a unified estimation strategy which combines a regression type formulation of sufficient dimension reduction methods and shrinkage estimation to produce sparse and accurate solutions the method can be applied to most existing sufficient dimension reduction methods such as sliced inverse regression sliced average variance estimation and principal hessian directions we demonstrate the effectiveness of the proposed method by both simulations and real data analysis

critical behaviour of the two dimensional potts model with a continuous number of states a finite size scaling analysis we investigate the critical behaviour of the two dimensional q state potts model using finite size scaling and transfer matrix methods for the continuous transition range 0 these results for continuous q were obtained from a transfer matrix constructed for a generalized whitney polynomial representing the potts models an effective algorithm to compute the dominant eigenvalues of this essentially nonsymmetric transfer matrix is developed

what is the goal of sensory coding a number of recent attempts have been made to describe early sensory coding in terms of a general information processing strategy in this paper two strategies are contrasted both strategies take advantage of the redundancy in the environment to produce more effective representations the first is described as a compact coding scheme a compact code performs a transform that allows the input to be represented with a reduced number of vectors cells with minimal rms error this approach has recently become popular in the neural network literature and is related to a process called principal components analysis pca a number of recent papers have suggested that the optimal compact code for representing natural scenes will have units with receptive field profiles much like those found in the retina and primary visual cortex however in this paper it is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway in contrast it is proposed that the visual system is near to optimal in representing natural scenes only if optimality is defined in terms of sparse distributed coding in a sparse distributed code all cells in the code have an equal response probability across the class of images but have a low response probability for any single image in such a code the dimensionality is not reduced rather the redundancy of the input is transformed into the redundancy of the firing pattern of cells it is proposed that the signature for a sparse code is found in the fourth moment of the response distribution i e the kurtosis in measurements with 55 calibrated natural scenes the kurtosis was found to peak when the bandwidths of the visual code matched those of cells in the mammalian visual cortex codes resembling wavelet transforms are proposed to be effective because the response histograms of such codes are sparse i e show high kurtosis when presented with natural scenes it is proposed that the structure of the image that allows sparse coding is found in the phase spectrum of the image it is suggested that natural scenes to a first approximation can be considered as a sum of self similar local functions the inverse of a wavelet possible reasons for why sensory systems would evolve toward sparse coding are presented

path consistent model selection in additive risk model via lasso as a flexible alternative to the cox model the additive risk model assumes that the hazard function is the sum of the baseline hazard and a regression function of covariates for right censored survival data when variable selection is needed along with model estimation we propose a path consistent model selector using a modified lasso approach under the additive risk model assumption we show that the proposed estimator possesses the oracle variable selection and estimation property applications of the proposed approach to three right censored survival data sets show that the proposed modified lasso yields parsimonious models with satisfactory estimation and prediction results

exact multicritical behaviour of the potts model a two dimensional q state potts model with vacancies and four spin interactions is studied the parameter space of the model contains a critical and a tricritical manifold moreover for 0 or q or 9 4 a multicritical point is found which is the locus where the tricritical transition changes from first to second order at this multicritical point which can be located exactly the model is solvable the authors compute the value of the central charge and a number of critical exponents

a fast tracking algorithm for generalized lars lasso this letter gives an efficient algorithm for tracking the solution curve of sparse logistic regression with respect to the regularization parameter the algorithm is based on approximating the logistic regression loss by a piecewise quadratic function using rosset and zhu s path tracking algorithm on the approximate problem and then applying a correction to get to the true path application of the algorithm to text classification and sparse kernel logistic regression shows that the algorithm is efficient

slow feature analysis unsupervised learning of invariances invariant features of temporally varying signals are useful for analysis and classification slow feature analysis sfa is a new method for learning invariant or slowly varying features from a vectorial input signal it is based on a nonlinear expansion of the input signal and application of principal component analysis to this expanded signal and its time derivative it is guaranteed to find the optimal solution within a family of functions directly and can learn to extract a large number of decorrelated features which are ordered by their degree of invariance sfa can be applied hierarchically to process high dimensional input signals and extract complex features sfa is applied first to complex cell tuning properties based on simple cell output including disparity and motion then more complicated input output functions are learned by repeated application of sfa finally a hierarchical network of sfa modules is presented as a simple model of the visual system the same unstructured network can learn translation size rotation contrast or to a lesser degree illumination invariance for one dimensional objects depending on only the training stimulus surprisingly only a few training objects suffice to achieve good generalization to new objects the generated representation is suitable for object recognition performance degrades if the network is trained to learn multiple invariances simultaneously

sample size determination for logistic regression the problem of sample size estimation is important in medical applications especially in cases of expensive measurements of immune biomarkers this paper describes the problem of logistic regression analysis with the sample size determination algorithms namely the methods of univariate statistics logistics regression cross validation and bayesian inference the authors treating the regression model parameters as a multivariate variable propose to estimate the sample size using the distance between parameter distribution functions on cross validated data sets herewith the authors give a new contribution to data mining and statistical learning supported by applied mathematics

forward and backward least angle regression for nonlinear system identification a forward and backward least angle regression lar algorithm is proposed to construct the nonlinear autoregressive model with exogenous inputs narx that is widely used to describe a large class of nonlinear dynamic systems the main objective of this paper is to improve model sparsity and generalization performance of the original forward lar algorithm this is achieved by introducing a replacement scheme using an additional backward lar stage the backward stage replaces insignificant model terms selected by forward lar with more significant ones leading to an improved model in terms of the model compactness and performance a numerical example to construct four types of narx models namely polynomials radial basis function rbf networks neuro fuzzy and wavelet networks is presented to illustrate the effectiveness of the proposed technique in comparison with some popular methods

two dimensional o n model in a staggered field nienhuis truncated o n model gives rise to a model of self avoiding loops on the hexagonal lattice each loop having a fugacity of n we study such loops subjected to a particular kind of staggered field w which for n has the geometrical effect of breaking the three phase coexistence linked to the three colourability of the lattice faces we show that at t 0 for w 1 the model flows to the ferromagnetic potts model with q n2 states with an associated fragmentation of the target space of the coulomb gas for t 0 there is a competition between t and w which gives rise to multicritical versions of the dense and dilute loop universality classes via an exact mapping and numerical results we establish that the latter two critical branches coincide with those found earlier in the o n model on the triangular lattice using transfer matrix studies we have found the renormalization group flows in the full phase diagram in the t w plane with fixed n superposing three copies of such hexagonal lattice loop models with staggered fields produces a variety of one or three species fully packed loop models on the triangular lattice with certain geometrical constraints possessing integer central charges 0 c 6 in particular we show that benjamini and schramm s rgb loops have fractal dimension df 3 2

the design and use of steerable filters the authors present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters allowing one to adaptively steer a filter to any orientation and to determine analytically the filter output as a function of orientation steerable filters may be designed in quadrature pairs to allow adaptive control over phase as well as orientation the authors show how to design and steer the filters and present examples of their use in the analysis of orientation and phase angularly adaptive filtering edge detection and shape from shading one can also build a self similar steerable pyramid representation the same concepts can be generalized to the design of 3 d steerable filters

lattice derivation of modular invariant partition functions on the torus the author expresses the partition functions of the ade lattice models on the torus in terms of partition function of six vertex models and which can be computed in the continuum limit thus the author recovers the partition function of minimal models with c 1 and new ones with c 1

variable inclusion and shrinkage algorithms the lasso is a popular and computationally efficient procedure for automatically performing both variable selection and coefficient shrinkage on linear regression models one limitation of the lasso is that the same tuning parameter is used for both variable selection and shrinkage as a result it typically ends up selecting a model with too many variables to prevent overshrinkage of the regression coefficients we suggest an improved class of methods called variable inclusion and shrinkage algorithms visa our approach is capable of selecting sparse models while avoiding overshrinkage problems and uses a path algorithm and so also is computationally efficient we show through extensive simulations that visa significantly outperforms the lasso and also provides improvements over more recent procedures such as the dantzig selector relaxed lasso and adaptive lasso in addition we provide theoretical justification for visa in terms of nonasymptotic bounds on the estimation error that suggest it shoul

order parameters of the dilute a models the free energy and local height probabilities of the dilute a models with borken 2 symmetry are calculated analytically using inversion and corner transfer matrix methods these models possess four critical branches the first two branches provide new realizations of the unitary minimal series and the other two branches give a direct product of this series with an ising model we identify the integrable perturbations which move the dilute a models away from the critical limit generalized order parameters are defined and their critical exponents extracted the associated conformal weights are found to occur on the diagonal of the relevant kac table in an appropriate regime the dilute a3 model lies in the universality class of the ising model in a magnetic field in this case we obtain the magnetic exponent 15 directly without the use of scaling relations

triangular lattice potts models in this paper we study the 3 state potts model on the triangular lattice which has two and three site interactions using a peierls argument we obtain a rigorous bound on the transition temperature thereby disproving a conjecture on the location of its critical point low temperature series are generated and analyzed for three particular choices of the coupling constants a phase diagram is then drawn on the basis of these considerations our analysis indicates that the antiferromagnetic transition and the transition along the coexistence line are of first order implying the existence of a multicritical point in the ferromagnetic region relation of the triangularq state potts model with other lattice statistical problems is also discussed in particular an ashkin teller model and the hard hexagon lattice gas solved by baxter emerge as special cases in appropriate limits

critical properties of the izergin korepin and solvable o n models and their related quantum spin chains the central charge of the izergin korepin model the corresponding quantum spin chain and the o n model is calculated analytically via the bethe ansatz the calculation extends a technique recently developed for the zamolodchikov fateev model in addition critical exponents and the central charge for these models are obtained from numerical solutions of bethe ansatz equations for finite systems as a physical application the authors find the exponents v 12 23 and gamma 53 46 for the theta transition of polymers in two dimensions

graphical approach to the nonintersecting string model star triangle equation inversion relation and exact solution abstract a general q state nonintersecting string model for q 2 is studied and solved using a graphical approach this model of which special cases were first introduced by strogonov and schultz has recently drawn further attention because of its relation to potts models in its simplest version a uniform separable nonintersecting string model on a quadratic lattice it corresponds to a q2 state potts model with potts spins on every other lattice face we first formulate the related star triangle equation which we solve in terms of a line variable parametrization such a parametrization of a solution of the star triangle equation leads to a simple and direct graphical derivation of an inversion relation for the partition function our graphical analysis also shows that the inversion relation holds if certain boundary effects can be neglected as we shall give a special finite lattice from which the inversion relation can be read off immediately this relation is next solved under appropriate analyticity assumptions again using a simple and direct approach and the result is applied to obtain exact solutions for specific string models

splines for financial volatility summary we propose a flexible generalized auto regressive conditional heteroscedasticity type of model for the prediction of volatility in financial time series the approach relies on the idea of using multivariate b splines of lagged observations and volatilities estimation of such a b spline basis expansion is constructed within the likelihood framework for non gaussian observations as the dimension of the b spline basis is large i e many parameters we use regularized and sparse model fitting with a boosting algorithm our method is computationally attractive and feasible for large dimensions we demonstrate its strong predictive potential for financial volatility on simulated and real data and also in comparison with other approaches and we present some supporting asymptotic arguments

the o n model on the triangular lattice to find critical points of o n models on the triangular lattice we apply two methods first we investigate the yang baxter equations on the triangular lattice we find only solvable points directly related to those for the square lattice second we construct intersections with the potts model this yields eight branches of critical points parametrized by n we establish the equivalence of these branches with the known critical points of the o n model on the square lattice transfer matrix calculations are performed to obtain the conformal anomaly and the thermal exponent of these branches these results include a numerical analysis of a q 3 potts tricritical point we find analytic relations between potts and o n models as well as among o n models with different values of n and among potts models with different values of q

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

critical behaviour and conformal anomaly of the o n model on the square lattice finite size scaling and transfer matrix techniques are used to determine the conformal anomaly and critical exponents of o n models on the square lattice these calculations were performed on five branches of critical points parametrised by n the results for two of the branches agree with the known universal properties of the o n model as derived for the honeycomb lattice finite size convergence is poor for most of the third branch so that in this case the classification in terms of known models is uncertain the authors data show that the fourth branch can be interpreted in terms of a combination of a low temperature o n and a critical ising model finally the numerical results for the o n model on the remaining branch agree accurately with the known results for the low temperature o n 1 model they prove this equivalence by means of an exact mapping

modeling the shape of the scene a holistic representation of the spatial envelope in this paper we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions the procedure is based on a very low dimensional representation of the scene that we term the spatial envelope we propose a set of perceptual dimensions naturalness openness roughness expansion ruggedness that represent the dominant spatial structure of a scene then we show that these dimensions may be reliably estimated using spectral and coarsely localized information the model generates a multidimensional space in which scenes sharing membership in semantic categories e g streets highways coasts are projected closed together the performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category

a generalized dantzig selector with shrinkage tuning summary the dantzig selector performs variable selection and model fitting in linear regression it uses an l 1 penalty to shrink the regression coefficients towards zero in a similar fashion to the lasso while both the lasso and dantzig selector potentially do a good job of selecting the correct variables they tend to overshrink the final coefficients this results in an unfortunate trade off one can either select a high shrinkage tuning parameter that produces an accurate model but poor coefficient estimates or a low shrinkage parameter that produces more accurate coefficients but includes many irrelevant variables we extend the dantzig selector to fit generalized linear models while eliminating overshrinkage of the coefficient estimates and develop a computationally efficient algorithm similar in nature to least angle regression to compute the entire path of coefficient estimates a simulation study illustrates the advantages of our approach relative to others we apply the methodology to two datasets

adaptive lasso for cox s proportional hazards model summary we investigate the variable selection problem for cox s proportional hazards model and propose a unified model selection and estimation procedure with desired theoretical properties and computational convenience the new method is based on a penalized log partial likelihood with the adaptively weighted l1 penalty on regression coefficients providing what we call the adaptive lasso estimator the method incorporates different penalties for different coefficients unimportant variables receive larger penalties than important ones so that important variables tend to be retained in the selection process whereas unimportant variables are more likely to be dropped theoretical properties such as consistency and rate of convergence of the estimator are studied we also show that with proper choice of regularization parameters the proposed estimator has the oracle properties the convex optimization nature of the method leads to an efficient algorithm both simulated and real examples show that the method performs competitively

feature screening via distance correlation learning this article is concerned with screening features in ultrahigh dimensional data analysis which has become increasingly important in diverse scientific fields we develop a sure independence screening procedure based on the distance correlation dc sis the dc sis can be implemented as easily as the sure independence screening sis procedure based on the pearson correlation proposed by fan and lv however the dc sis can significantly improve the sis fan and lv established the sure screening property for the sis based on linear models but the sure screening property is valid for the dc sis under more general settings including linear models furthermore the implementation of the dc sis does not require model specification e g linear model or generalized linear model for responses or predictors this is a very appealing property in ultrahigh dimensional data analysis moreover the dc sis can be used directly to screen grouped predictor variables and multivariate response variables we establish

a group bridge approach for variable selection in multiple regression problems when covariates can be naturally grouped it is important to carry out feature selection at the group and within group individual variable levels simultaneously the existing methods including the lasso and group lasso are designed for either variable selection or group selection but not for both we propose a group bridge approach that is capable of simultaneous selection at both the group and within group individual variable levels the proposed approach is a penalized regularization method that uses a specially designed group bridge penalty it has the oracle group selection property in that it can correctly select important groups with probability converging to one in contrast the group lasso and group least angle regression methods in general do not possess such an oracle property in group selection simulation studies indicate that the group bridge has superior performance in group and individual variable selection relative to several existing methods

dynamic scene recognition with complementary spatiotemporal features this paper presents dynamically pooled complementary features dpcf a unified approach to dynamic scene recognition that analyzes a short video clip in terms of its spatial temporal and color properties the complementarity of these properties is preserved through all main steps of processing including primitive feature extraction coding and pooling in the feature extraction step spatial orientations capture static appearance spatiotemporal oriented energies capture image dynamics and color statistics capture chromatic information subsequently primitive features are encoded into a mid level representation that has been learned for the task of dynamic scene recognition finally a novel dynamic spacetime pyramid is introduced this dynamic pooling approach can handle both global as well as local motion by adapting to the temporal structure as guided by pooling energies the resulting system provides online recognition of dynamic scenes that is thoroughly evaluated on the two current benchmark datasets and yields best results to date on both datasets in depth analysis reveals the benefits of explicitly modeling feature complementarity in combination with the dynamic spacetime pyramid indicating that this unified approach should be well suited to many areas of video analysis

simultaneous variable selection we propose a new method for selecting a common subset of explanatory variables where the aim is to model several response variables the idea is a natural extension of the lasso technique proposed by tibshirani 1996 and is based on the joint residual sum of squares while constraining the parameter estimates to lie within a suitable polyhedral region the properties of the resulting convex programming problem are analyzed for the special case of an orthonormal design for the general case we develop an efficient interior point algorithm the method is illustrated on a dataset with infrared spectrometry measurements on 14 qualitatively different but correlated responses using 770 wavelengths the aim is to select a subset of the wavelengths suitable for use as predictors for as many of the responses as possible

a note on path based variable selection in the penalized proportional hazards model summary we propose an efficient and adaptive shrinkage method for variable selection in the cox model the method constructs a piecewise linear regularization path connecting the maximum partial likelihood estimator and the origin then a model is selected along the path we show that the constructed path is adaptive in the sense that with a proper choice of regularization parameter the fitted model works as well as if the true underlying submodel were given in advance a modified algorithm of the least angle regression type efficiently computes the entire regularization path of the new estimator furthermore we show that with a proper choice of shrinkage parameter the method is consistent in variable selection and efficient in estimation simulation shows that the new method tends to outperform the lasso and the smoothly clipped absolute deviation estimators with moderate samples we apply the methodology to data concerning nursing homes

what does the retina know about natural scenes by examining the experimental data on the statistical properties of natural scenes together with retinal contrast sensitivity data we arrive at a first principle theoretical hypothesis for the purpose of retinal processing and its relationship to an animal s environment we argue that the retinal goal is to transform the visual input as much as possible into a statistically independent basis as the first step in creating a redundancy reduced representation in the cortex as suggested by barlow the extent of this whitening of the input is limited however by the need to suppress input noise our explicit theoretical solutions for the retinal filters also show a simple dependence on mean stimulus luminance they predict an approximate weber law at low spatial frequencies and a de vries rose law at high frequencies assuming that the dominant source of noise is quantum we generate a family of contrast sensitivity curves as a function of mean luminance this family is compared to psychophysical data

nonintersecting string model and graphical approach equivalence with a potts model using a graphical method we establish the exact equivalence of the partition function of aq state nonintersecting string nis model on an arbitrary planar even valenced lattice with that of a q2 state potts model on a related lattice the nis model considered in this paper is one in which the vertex weights are expressible as sums of those of basic vertex types and the resulting potts model generally has multispin interactions for the square and kagome lattices this leads to the equivalence of a staggered nis model with potts models with anisotropic pair interactions indicating that these nis models have a first order transition forq 2 for the triangular lattice the nis model turns out to be the five vertex model of wu and lin and it relates to a potts model with two and three site interactions the most general model we discuss is an oriented nis model which contains the six vertex model and the nis models of stroganov and schultz as special cases