Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the original IVFADC as well as all of its variants, after residuals are computed, the second production quantization step is completely independent of the first vector quantization step. In this work, we seek to exploit the connection between these two steps when we perform non-exhaustive search. More specifically, we decompose a residual vector locally into two orthogonal components and perform uniform quantization and multiscale quantization to each component respectively. The proposed method, called local orthogonal decomposition, combined with multiscale quantization consistently achieves higher recall than previous methods under the same bitrates. We conduct comprehensive experiments on large scale datasets as well as detailed ablation tests, demonstrating effectiveness of our method.
Rotations and codebooks are often applied in IVFADC variations, but there are significant costs associated with them. In the most extreme cases, Locally Optimized Product Quantization (LOPQ) @cite_15 learns a separate rotation matrix and codebook for each partition. This leads to an extra memory cost of @math and @math more multiplications for each query at search time. where @math is the number of VQ partitions we search. When @math and @math increase, the overhead become quickly noticeable and may become even more expensive than ADC itself. For example, when @math , @math , @math , performing the rotation once is as expensive as performing 6,400 ADC computations under an optimized implementation. In practice, it is often desirable to avoid per partition rotation or codebooks, but learn global codebooks and rotation.
@cite_15
2133995768
In many computer vision applications, a desired model of some type is computed by minimizing a cost function based on several measurements. Typically, one may compute the model that minimizes the @math cost, that is the sum of squares of measurement errors with respect to the model. However, the @math solution which minimizes the sum of the @math th power of errors usually gives more robust results in the presence of outliers for some values of @math , for example, @math . The Weiszfeld algorithm is a classic algorithm for finding the geometric @math mean of a set of points in Euclidean space. It is provably optimal and requires neither differentiation, nor line search. The Weiszfeld algorithm has also been generalized to find the @math mean of a set of points on a Riemannian manifold of non-negative curvature. This paper shows that the Weiszfeld approach may be extended to a wide variety of problems to find an @math mean for @math , while maintaining simplicity and provable convergence. We apply this problem to both single-rotation averaging (under which the algorithm provably finds the global @math optimum) and multiple rotation averaging (for which no such proof exists). Experimental results of @math optimization for rotations show the improved reliability and robustness compared to @math optimization. The recent years have witnessed the emerging of vector quantization (VQ) techniques for efficient similarity search. VQ partitions the feature space into a set of codewords and encodes data points as integer indices using the codewords. Then the distance between data points can be efficiently approximated by simple memory lookup operations. By the compact quantization, the storage cost, and searching complexity are significantly reduced, thereby facilitating efficient large-scale similarity search. However, the performance of several celebrated VQ approaches degrades significantly when dealing with noisy data. In addition, it can barely facilitate a wide range of applications as the distortion measurement only limits to @math norm . To address the shortcomings of the squared Euclidean ( @math norm) loss function employed by the VQ approaches, in this paper, we propose a novel robust and general VQ framework, named RGVQ, to enhance both robustness and generalization of VQ approaches. Specifically, a @math -norm loss function is proposed to conduct the @math -norm similarity search, rather than the @math norm search, and the @math -th order loss is used to enhance the robustness. Despite the fact that changing the loss function to @math norm makes VQ approaches more robust and generic, it brings us a challenge that a non-smooth and non-convex orthogonality constrained @math - norm function has to be minimized. To solve this problem, we propose a novel and efficient optimization scheme and specify it to VQ approaches and theoretically prove its convergence. Extensive experiments on benchmark data sets demonstrate that the proposed RGVQ is better than the original VQ for several approaches, especially when searching similarity in noisy data. A set X of points in @math is (k,b)-clusterable if X can be partitioned into k subsets (clusters) so that the diameter (alternatively, the radius) of each cluster is at most b. We present algorithms that, by sampling from a set X, distinguish between the case that X is (k,b)-clusterable and the case that X is @math -far from being (k,b')-clusterable for any given @math and for @math . By @math -far from being (k,b')-clusterable we mean that more than @math points should be removed from X so that it becomes (k,b')-clusterable. We give algorithms for a variety of cost measures that use a sample of size independent of |X| and polynomial in k and @math . Our algorithms can also be used to find approximately good clusterings. Namely, these are clusterings of all but an @math -fraction of the points in X that have optimal (or close to optimal) cost. The benefit of our algorithms is that they construct an implicit representation of such clusterings in time independent of |X|. That is, without actually having to partition all points in X, the implicit representation can be used to answer queries concerning the cluster to which any given point belongs. We study the problem of constructing universal Steiner trees for undirected graphs. Given a graph @math and a root node @math , we seek a single spanning tree @math of minimum stretch , where the stretch of @math is defined to be the maximum ratio, over all terminal sets @math , of the cost of the minimal sub-tree @math of @math that connects @math to @math to the cost of an optimal Steiner tree connecting @math to @math in @math . Universal Steiner trees (USTs) are important for data aggregation problems where computing the Steiner tree from scratch for every input instance of terminals is costly, as for example in low energy sensor network applications. We provide a polynomial time UST construction for general graphs with @math -stretch. We also give a polynomial time polylogarithmic-stretch construction for minor-free graphs. One basic building block of our algorithms is a hierarchy of graph partitions, each of which guarantees small strong diameter for each cluster and bounded neighbourhood intersections for each node. We show close connections between the problems of constructing USTs and building such graph partitions. Our construction of partition hierarchies for general graphs is based on an iterative cluster merging procedure, while the one for minor-free graphs is based on a separator theorem for such graphs and the solution to a cluster aggregation problem that may be of independent interest even for general graphs. To our knowledge, this is the first sub polynomial-stretch ( @math for any @math ) UST construction for general graphs, and the first polylogarithmic-stretch UST construction for minor-free graphs.