aggregating local image descriptors into compact codes this paper addresses the problem of large scale image search three constraints have to be taken into account search accuracy efficiency and memory usage we first present and evaluate different ways of aggregating local image descriptors into a vector and show that the fisher kernel achieves better performance than the reference bag of visual words approach for any given vector dimension we then jointly optimize dimensionality reduction and indexing in order to obtain a precise vector comparison as well as a compact representation the evaluation shows that the image representation can be reduced to a few dozen bytes while preserving high accuracy searching a 100 million image data set takes about 250 ms on one processor core

spectral gmm estimation of continuous time processes abstract this paper derives a methodology for the estimation of continuous time stochastic models based on the characteristic function the estimation method does not require discretization of the stochastic process and it is simple to apply in practice the method is essentially generalized method of moments on the complex plane hence it shares the efficiency and distribution properties of gmm estimators we illustrate the method with some applications to relevant estimation problems in continuous time finance we estimate a model of stochastic volatility a jump diffusion model with constant volatility and a model that nests both the stochastic volatility model and the jump diffusion model we find that negative jumps are important to explain skewness and asymmetry in excess kurtosis of the stock return distribution while stochastic volatility is important to capture the overall level of this kurtosis positive jumps are not statistically significant once we allow for stochastic volatility in the model we also estimate a non affine model of stochastic volatility and find that the power of the diffusion coefficient appears to be between one and two rather than the value of one half that leads to the standard affine stochastic volatility model however we find that including jumps into this non affine stochastic volatility model reduces the power of the diffusion coefficient to one half finally we offer an explanation for the observation that the estimate of persistence in stochastic volatility increases dramatically as the frequency of the observed data falls based on a multiple factor stochastic volatility model

efficient estimation of general dynamic models with a continuum of moment conditions there are two difficulties with the implementation of the characteristic function based estimators first the optimal instrument yielding the ml efficiency depends on the unknown probability density function second the need to use a large set of moment conditions leads to the singularity of the covariance matrix we resolve the two problems in the framework of gmm with a continuum of moment conditions a new optimal instrument relies on the double indexing and as a result has a simple exponential form the singularity problem is addressed via a penalization term we introduce hac type estimators for non markov models a simulated method of moments is proposed for non analytical cases

trajectory based place recognition for efficient large scale localization place recognition is a core competency for any visual simultaneous localization and mapping system identifying previously visited places enables the creation of globally accurate maps robust relocalization and multi user mapping to match one place to another most state of the art approaches must decide a priori what constitutes a place often in terms of how many consecutive views should overlap or how many consecutive images should be considered together unfortunately such threshold dependencies limit their generality to different types of scenes in this paper we present a placeless place recognition algorithm using a novel match density estimation technique that avoids heuristically discretizing the space instead our approach considers place recognition as a problem of continuous matching between image streams automatically discovering regions of high match density that represent overlapping trajectory segments the algorithm uses well studied statistical tests to identify the relevant matching regions which are subsequently passed to an absolute pose algorithm to recover the geometric alignment we demonstrate the efficiency and accuracy of our methodology on three outdoor sequences including a comprehensive evaluation against ground truth from publicly available datasets that shows our approach outperforms several state of the art algorithms for place recognition furthermore we compare our overall algorithm to the currently best performing system for global localization and show how we outperform the approach on challenging indoor and outdoor datasets

asymmetric distances for binary embeddings in large scale query by example retrieval embedding image signatures in a binary space offers two benefits data compression and search efficiency while most embedding algorithms binarize both query and database signatures it has been noted that this is not strictly a requirement indeed asymmetric schemes that binarize the database signatures but not the query still enjoy the same two benefits but may provide superior accuracy in this work we propose two general asymmetric distances that are applicable to a wide variety of embedding techniques including locality sensitive hashing lsh locality sensitive binary codes lsbc spectral hashing sh pca embedding pcae pcae with random rotations pcae rr and pcae with iterative quantization pcae itq we experiment on four public benchmarks containing up to 1m images and show that the proposed asymmetric distances consistently lead to large improvements over the symmetric hamming distance for all binary embedding techniques

hypothesis testing in time series via the empirical characteristic function a generalized spectral density approach abstract the standardized spectral density completely describes serial dependence of a gaussian process for non gaussian processes however it may become an inappropriate analytic tool because it misses the nonlinear processes with zero autocorrelation by generalizing the concept of the standardized spectral density i propose a new spectral tool suitable for both linear and nonlinear time series analysis the generalized spectral density is indexed by frequency and a pair of auxiliary parameters it is well defined for both continuous and discrete random variables and requires no moment condition introduction of the auxiliary parameters renders the spectrum able to capture all pairwise dependencies including those with zero autocorrelation the standardized spectral density can be derived by properly differentiating the generalized spectral density with respect to the auxiliary parameters at the origin the consistency of a class of parzen s kernel type estimators for the generalized spectral dens

a conditional kolmogorov test this paper introduces a conditional kolmogorov test of model specification for parametric models with covariates regressors the test is an extension of the kolmogorov test of goodness of fit for distribution functions the test is shown to have power against 1 n local alternatives and all fixed alternatives to the null hypothesis a parametric bootstrap procedure is used to obtain critical values for the test

optimized product quantization product quantization pq is an effective vector quantization method a product quantizer can generate an exponentially large codebook at very low memory time cost the essence of pq is to decompose the high dimensional vector space into the cartesian product of subspaces and then quantize these subspaces separately the optimal space decomposition is important for the pq performance but still remains an unaddressed issue in this paper we optimize pq by minimizing quantization distortions w r t the space decomposition and the quantization codebooks we present two novel solutions to this challenging optimization problem the first solution iteratively solves two simpler sub problems the second solution is based on a gaussian assumption and provides theoretical analysis of the optimality we evaluate our optimized product quantizers in three applications i compact encoding for exhaustive ranking 1 ii building inverted multi indexing for non exhaustive search 2 and iii compacting image representations for image retrieval 3 in all applications our optimized product quantizers outperform existing solutions

estimation of affine asset pricing models using the empirical characteristic function abstract the known functional form of the conditional characteristic function ccf of discretely sampled observations from an affine diffusion is used to develop computationally tractable and asymptotically efficient estimators of the parameters of affine diffusions and of asset pricing models in which the state vectors follow affine diffusions both time domain estimators based on fourier inversion of the ccf and frequency domain estimators based directly on the ccf are constructed a method of moments estimator based on the ccf is shown to approximate the efficiency of maximum likelihood for affine diffusion and asset pricing models

four one shot learners for regular tree languages and their polynomial characterizability we survey two existing algorithms for the inference of finite state tree automata from membership queries and a finite positive sample or equivalence queries and we suggest a reformulation of one of them which we deem necessary to ensure its termination we present two algorithms for the same two settings when the underlying description is not a deterministic but a residual canonical finite state automaton to this end we adapt all necessary notions to the residual tree case from our completed perspective we discuss the terms on which those four algorithms can be considered to be of polynomial complexity and also where there may be hidden exponentiality

a consistent characteristic function based test for conditional independence abstract y is conditionally independent of z given x if pr f y x z f y x 1 for all y on its support where f denotes the conditional density of y given x z or x this paper proposes a nonparametric test of conditional independence based on the notion that two conditional distributions are equal if and only if the corresponding conditional characteristic functions are equal we extend the test of su and white 2005 a hellinger metric nonparametric test for conditional independence discussion paper department of economics ucsd in two directions 1 our test is less sensitive to the choice of bandwidth sequences 2 our test has power against deviations on the full support of the density of x y z we establish asymptotic normality for our test statistic under weak data dependence conditions simulation results suggest that the test is well behaved in finite samples applications to stock market data indicate that our test can reveal some interesting nonlinear dependence that a traditional linear granger causality test fails to detect

the ordinal relation preserving binary codes hashing algorithm has been widely used for efficient approximate nearest neighbor ann search learning optimal hashing functions has been given focus and it is still a challenge this paper aims to effectively and efficiently generate relative similarity preserving binary codes most existing hashing methods try to preserve the locality similarity by preserving direct distance similarity while ignoring the relative similarity which advantages in ann search in this paper this issue is solved by proposing the relative error which emphasizes that the ordinal relations in hamming space and euclidean space should be consistent with each other we learn hashing projection functions via two steps the first step adopts the lookup based mechanism to find the optimal binary codes of training data which can preserve the relative similarity and simultaneously adapt to data distribution the binary codes in the first step are considered as supervision information in the second step the objective of the second step is to learn hashing projection functions which can efficiently regenerate the binary codes in the first step aim to be in accordance with the property of data distribution the hyper internal tangent planes of two specified spheres are chosen as hashing projection functions assisted by these projection functions the time complexity of encoding process is greatly reduced experimental results on four public data sets demonstrate that our method outperforms many other state of the art methods author highlightsan effective and efficient binary coding framework which enjoys the merits of lookup based and hamming based methods is proposed relative error is proposed to preserve the relative sensitivity the effective tangent planes are adopted as hashing functions to avoid computing the complex classification problem the encoding error guarantees that the selected hashing functions have good performance

bilinear optimized product quantization for scalable visual content analysis product quantization pq has been recognized as a useful technique to encode visual feature vectors into compact codes to reduce both the storage and computation cost recent advances in retrieval and vision tasks indicate that high dimensional descriptors are critical to ensuring high accuracy on large scale data sets however optimizing pq codes with high dimensional data is extremely time consuming and memory consuming to solve this problem in this paper we present a novel pq method based on bilinear projection which can well exploit the natural data structure and reduce the computational complexity specifically we learn a global bilinear projection for pq where we provide both non parametric and parametric solutions the non parametric solution does not need any data distribution assumption the parametric solution can avoid the problem of local optima caused by random initialization and enjoys a theoretical error bound besides we further extend this approach by learning locally bilinear projections to fit underlying data distributions we show by extensive experiments that our proposed method dubbed bilinear optimization product quantization achieves competitive retrieval and classification accuracies while having significant lower time and space complexities

distributed adaptive binary quantization for fast nearest neighbor search hashing has been proved an attractive technique for fast nearest neighbor search over big data compared with the projection based hashing methods prototype based ones own stronger power to generate discriminative binary codes for the data with complex intrinsic structure however existing prototype based methods such as spherical hashing and k means hashing still suffer from the ineffective coding that utilizes the complete binary codes in a hypercube to address this problem we propose an adaptive binary quantization abq method that learns a discriminative hash function with prototypes associated with small unique binary codes our alternating optimization adaptively discovers the prototype set and the code set of a varying size in an efficient way which together robustly approximate the data relations our method can be naturally generalized to the product space for long hash codes and enjoys the fast training linear to the number of the training data we further devise a distributed framework for the large scale learning which can significantly speed up the training of abq in the distributed environment that has been widely deployed in many areas nowadays the extensive experiments on four large scale up to 80 million data sets demonstrate that our method significantly outperforms state of the art hashing methods with up to 58 84 performance gains relatively

robust quantization for general similarity search the recent years have witnessed the emerging of vector quantization vq techniques for efficient similarity search vq partitions the feature space into a set of codewords and encodes data points as integer indices using the codewords then the distance between data points can be efficiently approximated by simple memory lookup operations by the compact quantization the storage cost and searching complexity are significantly reduced thereby facilitating efficient large scale similarity search however the performance of several celebrated vq approaches degrades significantly when dealing with noisy data in addition it can barely facilitate a wide range of applications as the distortion measurement only limits to ell 2 norm to address the shortcomings of the squared euclidean ell 2 2 norm loss function employed by the vq approaches in this paper we propose a novel robust and general vq framework named rgvq to enhance both robustness and generalization of vq approaches specifically a ell p q norm loss function is proposed to conduct the ell p norm similarity search rather than the ell 2 norm search and the q th order loss is used to enhance the robustness despite the fact that changing the loss function to ell p q norm makes vq approaches more robust and generic it brings us a challenge that a non smooth and non convex orthogonality constrained ell p q norm function has to be minimized to solve this problem we propose a novel and efficient optimization scheme and specify it to vq approaches and theoretically prove its convergence extensive experiments on benchmark data sets demonstrate that the proposed rgvq is better than the original vq for several approaches especially when searching similarity in noisy data

polynomial time learning of simple deterministic languages via queries and a representative sample we show that simple deterministic languages are polynomial time learnable via membership queries if the learner knows a special finite set of positive examples this finite set is called a representative sample and has been introduced by angluin inform control 51 1981 to show that regular languages are polynomial time learnable via membership queries if simple deterministic languages are learnable in polynomial time via membership and equivalence queries we can obtain a representative sample of a target language in polynomial time from a correct hypothesis thus our result implies that the polynomial time learning problem of simple deterministic languages via membership and equivalence queries is solvable if and only if we can find a representative sample in polynomial time via these queries we show the learnability of simple deterministic languages by giving a learning algorithm the algorithm at the first stage makes all possible candidate rules to generate the target language and a set of simple deterministic grammars which are little different each other then comparing them the algorithm eliminates inappropriate rules

nonparametric pricing of interest rate derivative securities we propose a nonparametric estimation procedure for continuous time stochastic models because prices of derivative securities depend crucially on the form of the instantaneous volatility of the underlying process we leave the volatility function unrestricted and estimate it nonparametrically only discrete data are used but the estimation procedure still does not rely on replacing the continuous time model by some discrete approximation instead the drift and volatility functions are forced to match the densities of the process we estimate the stochastic differential equation followed by the short term interest rate and compute nonparametric prices for bonds and bond options

nonparametric transition based tests for jump diffusions we develop a specification test for the transition density of a discretely sampled continuous time jump diffusion process based on a comparison of a nonparametric estimate of the transition density or distribution function with their corresponding parametric counterparts assumed by the null hypothesis as a special case our method applies to pure diffusions we provide a direct comparison of the two densities for an arbitrary specification of the null parametric model using three different discrepancy measures between the null and alternative transition density and distribution functions we establish the asymptotic null distributions of proposed test statistics and compute their power functions we investigate the finite sample properties through simulations and compare them with those of other tests this article has supplementary material online

a feasible method for optimization with orthogonality constraints minimization with orthogonality constraints e g x top x i and or spherical constraints e g vert x vert 2 1 has wide applications in polynomial optimization combinatorial optimization eigenvalue problems sparse pca p harmonic flows 1 bit compressive sensing matrix rank minimization etc these problems are difficult because the constraints are not only non convex but numerically expensive to preserve during iterations to deal with these difficulties we apply the cayley transform a crank nicolson like update scheme to preserve the constraints and based on it develop curvilinear search algorithms with lower flops compared to those based on projections and geodesics the efficiency of the proposed algorithms is demonstrated on a variety of test problems in particular for the maxcut problem it exactly solves a decomposition formulation for the sdp relaxation for polynomial optimization nearest correlation matrix estimation and extreme eigenvalue problems the proposed algorithms run very fast and return solutions no worse than those from their state of the art algorithms for the quadratic assignment problem a gap 0 842 to the best known solution on the largest problem tai256c in qaplib can be reached in 5 min on a typical laptop

reprojecting partially observed systems with application to interest rate diffusions abstract we introduce reprojection as a general purpose technique for characterizing the dynamic response of a partially observed nonlinear system to its observable history reprojection is the third step of a procedure wherein first data are summarized by projection onto a hermite series representation of the unconstrained transition density for observables second system parameters are estimated by minimum chi squared where the chi squared criterion is a quadratic form in the expected score of the projection and third the constraints on dynamics implied by the nonlinear system are imposed by projecting a long simulation of the estimated system onto a hermite series representation of the constrained transition density for observables the constrained transition density can be used to study the response of the system to its observable history we utilize the technique to assess the dynamics of several diffusion models for the short term interest rate that have been proposed and to compare them to a ne

the law of the euler scheme for stochastic differential equations i convergence rate of the distribution function we study the approximation problem ofef x t byef x t n where x t is the solution of a stochastic differential equation x t n is defined by the euler discretization scheme with stept n andf is a given function for smoothf s talay and tubaro have shown that the erroref x t f x t n can be expanded in powers of 1 n which permits to construct romberg extrapolation precedures to accelerate the convergence rate here we prove that the expansion exists also whenf is only supposed measurable and bounded under an additional nondegeneracy condition of hormander type for the infinitesimal generator of x t to obtain this result we use the stochastic variations calculus in the second part of this work we will consider the density of the law ofx t n and compare it to the density of the law ofx t

modeling the shape of the scene a holistic representation of the spatial envelope in this paper we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions the procedure is based on a very low dimensional representation of the scene that we term the spatial envelope we propose a set of perceptual dimensions naturalness openness roughness expansion ruggedness that represent the dominant spatial structure of a scene then we show that these dimensions may be reliably estimated using spectral and coarsely localized information the model generates a multidimensional space in which scenes sharing membership in semantic categories e g streets highways coasts are projected closed together the performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category

product quantization for nearest neighbor search this paper introduces a product quantization based approach for approximate nearest neighbor search the idea is to decompose the space into a cartesian product of low dimensional subspaces and to quantize each subspace separately a vector is represented by a short code composed of its subspace quantization indices the euclidean distance between two vectors can be efficiently estimated from their codes an asymmetric version increases precision as it computes the approximate distance between a vector and a code experimental results show that our approach searches for nearest neighbors efficiently in particular in combination with an inverted file system results for sift and gist image descriptors show excellent search accuracy outperforming three state of the art approaches the scalability of our approach is validated on a data set of two billion vectors

generalized spectral testing for multivariate continuous time models we develop an omnibus specification test for multivariate continuous time models using the conditional characteristic function which often has a convenient closed form or can be accurately approximated for many multivariate continuous time models in finance and economics the proposed test fully exploits the information in the joint conditional distribution of underlying economic processes and hence is expected to have good power in a multivariate context a class of easy to interpret diagnostic procedures is supplemented to gauge possible sources of model misspecification our tests are also applicable to discrete time distribution models simulation studies show that the tests provide reliable inference in finite samples

filtering via simulation auxiliary particle filters this article analyses the recently suggested particle approach to filtering time series we suggest that the algorithm is not robust to outliers for two reasons the design of the simulators and the use of the discrete support to represent the sequentially updating prior distribution here we tackle the first of these problems

a note on the number of queries needed to identify regular languages we consider the number of queries required to identify a regular set given an oracle for the set and some auxiliary information about the set if the auxiliary information is n the number of states of the canonical finite state acceptor for the language then the upper and lower bounds on the number of queries are exponential in n if the auxiliary information consists of a set of strings guaranteed to reach every live state of the canonical acceptor for the language then the upper and lower bounds are polynomial in n and the size of the given set of strings as a corollary the problem considered by pao and carr 1978 is shown to be solvable in a polynomial number of queries

a test for model specification of diffusion processes we propose a test for model specification of a parametric diffusion process based on a kernel estimation of the transitional density of the process the empirical likelihood is used to formulate a statistic for each kernel smoothing bandwidth which is effectively a studentized l2 distance between the kernel transitional density estimator and the parametric transitional density implied by the parametric process to reduce the sensitivity of the test on smoothing bandwidth choice the final test statistic is constructed by combining the empirical likelihood statistics over a set of smoothing bandwidths to better capture the finite sample distribution of the test statistic and data dependence the critical value of the test is obtained by a parametric bootstrap procedure properties of the test are evaluated asymptotically and numerically by simulation and by a real data example

alternative models for stock price dynamics abstract this paper evaluates the role of various volatility specifications such as multiple stochastic volatility sv factors and jump components in appropriate modeling of equity return distributions we use estimation technology that facilitates nonnested model comparisons and use a long data set which provides rich information about the conditional and unconditional distribution of returns we consider two broad families of models 1 the multifactor loglinear family and 2 the affine jump family both classes of models have attracted much attention in the derivatives and econometrics literatures there are various tradeoffs in considering such diverse specifications if pure diffusion sv models are chosen over jump diffusions it has important implications for hedging strategies if logarithmic models are chosen over affine ones it may seriously complicate option pricing comparing many different specifications of pure diffusion multifactor models and jump diffusion models we find that 1 log linear models have to be extended to two factors with feedback in the mean reverting factor 2 affine models have to have a jump in returns stochastic volatility or probably both models 1 and 2 are observationally equivalent on the data set in hand in either 1 or 2 the key is that the volatility can move violently as we obtain models with comparable empirical fit one must make a choice based on arguments other than statistical goodness of fit criteria the considerations include facility to price options to hedge and parsimony the affine specification with jumps in volatility might therefore be preferred because of the closed form derivatives prices

codewords distribution based optimal combination of equal average equal variance equal norm nearest neighbor fast search algorithm for vector quantization encoding vector quantization encoding requires expensive time to find the closest codeword through the codebook for each input vector a fast search algorithm for encoding is proposed in this paper the multilevel elimination criterion is still derived from the three features mean variance and norm inequalities constraints but the order of the three inequalities constraints instead of the predefined order like other conventional multilevel elimination criterion is optimized to speed up the encoding procedure in the proposed algorithm the elimination criterion at the first level is set to mean inequality constraint because of its narrower search width and the priority order at the second and or third level of variance and norm inequalities constraints is optimized based on the codewords distribution and the location of input vector in terms of the considered features the experimental results demonstrate the effectiveness of the proposed algorithm

learning regular sets from queries and counterexamples the problem of identifying an unknown regular set from examples of its members and nonmembers is addressed it is assumed that the regular set is presented by a minimamy adequate teacher which can answer membership queries about the set and can also test a conjecture and indicate whether it is equal to the unknown set and provide a counterexample if not a counterexample is a string in the symmetric difference of the correct set and the conjectured set a learning algorithm l is described that correctly learns any regular set from any minimally adequate teacher in time polynomial in the number of states of the minimum dfa for the set and the maximum length of any counterexample provided by the teacher it is shown that in a stochastic setting the ability of the teacher to test conjectures may be replaced by a random sampling oracle ex a polynomial time learning algorithm is shown for a particular problem of context free language identification