Package: n2
Lightweight approximateNearestNeighbor algorithm library written
in C++ (with Python/Go bindings).N2 stands for two N’s, which comes from 'ApproximateNearestNeighbor Algorithm'.Why N2 Was MadeBefore N2, there has been other great approximate nearest neighbor
libraries such asAnnoyandNMSLIB. However, each of them had
different strengths and weaknesses regarding usability, performance,
and etc. So, N2 has been developed aiming to bring the strengths of
existing aKNN libraries and supplement their weaknesses.FeaturesLightweight library which runs fast with large datasets.Good performance in terms of index build time, search speed,
and memory usage.Supports multi-core CPUs for index building.Supports a mmap feature by default to efficiently process large
index files.Supports Python/Go bindings.Supported Distance MetricsMetricDefinitiond(p,q)“angular”1 - cosθ1 - {sum(pi· qi) /
sqrt(sum(pi· pi) · sum(qi· qi))}“L2”squared L2sum{(pi- qi)2}“dot”dot productsum(pi· qi)N2 supports three distance metrics.
For “angular” and “L2”,d(distance) is defined such that the closer the vectors are,
the smallerdis. However for “dot”,dis defined such that the closer
the vectors are, the largerdis. You may be wondering why we defined
and implemented “dot” metric asplain dot productand not as(1 - dot product).
The rationale for this decision was to allow users to directly interpret thedvalue
returned from Hnsw search function as a dot product value.QuickstartInstall N2 with pip.$pipinstalln2Here is a python code snippet demonstrating how to use N2.importnumpyasnpfromn2importHnswIndexN,dim=10240,20samples=np.arange(N*dim).reshape(N,dim)index=HnswIndex(dim)forsampleinsamples:index.add_data(sample)index.build(m=5,n_threads=4)print(index.search_by_id(0,10))# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]Full DocumentationVisitn2.readthedocs.iofor full documentation.
The documentation site explains the following contents in detail.Installation Guidehow to build from source, etc.User Guide with Basic ExamplesPython InterfaceC++ InterfaceGo InterfaceBenchmarkdetailed explanation of how we performed benchmark experiemnts.PerformanceHere are the results of our benchmark experiments.You can also see benchmarks of various ANN libraries in Python atann-benchmarks.com.
Note that N2 version 0.1.6 is used inann-benchmarks.com(last checked on October 8th, 2020)
and we are continuing our efforts to improve N2 performance.Index Build TimeSearch SpeedMemory UsageReferencesY. A. Malkov and D. A. Yashunin, “Efficient and robust approximate
nearest neighbor search using hierarchical navigable small world
graphs,” CoRR, vol. abs/1603.09320, 2016. [Online].
Available:http://arxiv.org/abs/1603.09320NMSLIB:https://github.com/nmslib/nmslibAnnoy:https://github.com/spotify/annoyLicenseThis software is licensed under theApache 2 license, quoted below.Copyright 2017 Kakao Corp.http://www.kakaocorp.comLicensed under the Apache License, Version 2.0 (the “License”); you may
not use this project except in compliance with the License. You may
obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0.Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.