We demonstrate the first possibility of a sub-linear memory sketch for solving the approximate near-neighbor search problem. In particular, we develop an online sketching algorithm that can compress N vectors into a tiny sketch consisting of small arrays of counters whose size scales as O(N ^{b\log^2{N}) } ) in O(N^{b+1} \log^3{N}) time} , where b < 1 depending on the stability of the near-neighbor search. This sketch is sufficient to identify the top-v near-neighbors with high probability. To the best of our knowledge, this is the first near-neighbor search algorithm that breaks the linear memory (O(N)) barrier. We achieve sub-linear memory by combining advances in locality sensitive hashing (LSH) based estimation, especially the recently-published ACE algorithm, with compressed sensing and heavy hitter techniques. We provide strong theoretical guarantees; in particular, our analysis sheds new light on the memory-accuracy tradeoff in the near-neighbor search setting and the role of sparsity in compressed sensing , which could be of independent interest. We rigorously evaluate our framework, which we call RACE (Repeated ACE) data structures on a friend recommendation task on the Google plus graph with more than 100,000 high-dimensional vectors. RACE provides compression that is orders of magnitude better than the random projection based alternative, which is unsurprising given the theoretical advantage. We anticipate that RACE will enable both new theoretical perspectives on near-neighbor search and new methodologies for applications like high-speed data mining, internet-of-things (IoT), and beyond .
We present the first sublinear memory sketch which can be queried to find the v nearest neighbors in a dataset. Our online sketching algorithm can compress an N-element dataset to a sketch of size O(N \log^2{N}) } ^b \log^3{N) in O(N^{b+1} \log^3{N}) time} , where b < 1 when the query satisfies a data-dependent near-neighbor stability condition. We achieve data-dependent sublinear space by combining recent advances in locality sensitive hashing (LSH) -based estimators with compressed sensing . Our results shed new light on the memory-accuracy tradeoff for near-neighbor search . The techniques presented reveal a deep connection between the fundamental compressed sensing (or heavy hitters) recovery problem and near-neighbor search , leading to new insight for geometric search problems and implications for sketching algorithms .
R
demonstrate the first possibility of a sub-linear memory sketch for solving the approximate near-neighbor search problem. In particular, we develop an
present the first sublinear memory sketch which can be queried to find the v nearest neighbors in a dataset. Our
R
that can compress N vectors into a tiny sketch consisting of small arrays of counters whose size scales as
can compress an N-element dataset to a sketch of size
D
^{b
A
^b \log^3{N
R
depending on the stability of the
when the query satisfies a data-dependent
R
search. This sketch is sufficient to identify the top-v near-neighbors with high probability. To the best of our knowledge, this is the first near-neighbor search algorithm that breaks the linear memory (O(N)) barrier. We achieve sub-linear memory by combining
stability condition. We achieve data-dependent sublinear space by combining recent
R
based estimation, especially the recently-published ACE algorithm,
-based estimators
R
and heavy hitter techniques. We provide strong theoretical guarantees; in particular, our analysis sheds
. Our results shed
R
in the
for
R
setting and the role of sparsity in compressed sensing , which could be of independent interest. We rigorously evaluate our framework, which we call RACE (Repeated ACE) data structures on a friend recommendation task on the Google plus graph with more than 100,000 high-dimensional vectors. RACE provides compression that is orders of magnitude better than the random projection based alternative, which is unsurprising given the theoretical advantage. We anticipate that RACE will enable both new theoretical perspectives on
. The techniques presented reveal a deep connection between the fundamental compressed sensing (or heavy hitters) recovery problem and
R
and new methodologies for applications like high-speed data mining, internet-of-things (IoT), and beyond
, leading to new insight for geometric search problems and implications for sketching algorithms