Package: nearness
nearnessis a unified interface for (approximate) nearest neighbors search.Usingpip install nearnessonly installs the interface and does not add any concrete nearest
neighbors search implementation. The following implementations are available currently:AnnoyexposesAnnoyNeighborsAutoFaissexposesAutoFaissNeighborsFaissexposesFaissNeighborsPyGlassexposesGlassNeighborsHNSWLibexposesHNSWNeighborsJaxexposesJaxNeighborsNumpyexposesNumpyNeighborsScaNNexposesScannNeighborsSciPyexposesScipyNeighborsscikit-learnexposesSklearnNeighborsPyTorchexposesTorchNeighborsInstalling one of the above packages exposes the corresponding nearest neighbors implementation. For example,nearness.FaissNeighborsis available ifFaissis installed.Another option to install the underlying packages is to specify them as package extras, e.g.pip install nearness[faiss]installs the nearness withfaiss-cpu. If you require flexibility regarding
the specific version of the installed packages, it's recommended to install them explicitly.APIThe nearness API consists of a single class calledNearestNeighborswith the following methods.deffit(data:np.ndarray)->Self:"""Learn an index structure based on a matrix of points."""...defquery(point:np.ndarray,n_neighbors:int)->tuple[np.ndarray,np.ndarray]:"""Search ``n_neighbors`` for a single point, returning the indices and distances."""...defquery_batch(points:np.ndarray,n_neighbors:int)->tuple[np.ndarray,np.ndarray]:"""Search ``n_neighbors`` for a batch of points, returning the indices and distances."""...defsave(file:str|Path)->None:"""Save the state of the model using pickle such that it can be fully restored."""...defload(file:str|Path)->None:"""Load a model using pickle to fully restore the saved state."""...The interface to all methods is based onNumPyarrays, but implementations mightoverloadthe methods such that other data types are supported. For example,TorchNeighborssupports NumPy and
PyTorch arrays.The library additionally exports a globalconfigobject, of which the current state is passed to anyNearestNeighborsclass instantiation. Any modifications of a class-bound config is then specific to the class
and does not modify the global object.In addition to the global config, we treat all of the__init__arguments toNearestNeighborsasparametersof the class, automagically binding the parameters to an object before instantiation. We expose the
config and parameters of an object asobj.configandobj.parameters.Usage ExampleThe following example demonstrates how to use nearness given thatscikit-learnis installed.fromnearnessimportSklearnNeighborsfromsklearn.datasetsimportload_digitsfromsklearn.model_selectionimporttrain_test_splitX,_=load_digits(return_X_y=True)X_train,X_test=train_test_split(X)# create a brute force nearest neighbors modelmodel=SklearnNeighbors(algorithm="brute")model.fit(X_train)# query a single test pointidx,dist=model.query(X_test[0],n_neighbors=5)# query all test pointsidx_batch,dist_batch=model.query_batch(X_test,n_neighbors=5)# change the algorithm to a K-D tree and fit againmodel.parameters.algorithm="kd_tree"model.fit(X_train)# save the model to a filemodel.save("my_sklearn_model")# load the model from filekdtree_model=SklearnNeighbors.load("my_sklearn_model")# query again using the loaded modelkdtree_model.query(X_test[0],n_neighbors=5)Algorithm ImplementationTo define your ownNearestNeighborsalgorithm it is only necessary to implement above specifiedfitandquerymethods. By default,query_batchuses a joblib to process a batch of queries in a threadpool, but most of
the time you'd want to implementquery_batchon your own for improved efficiency.The following example illustrates the concepts ofconfigandparameters.classMyNearestNeighbors(NearestNeighbors):# only keyword-only arguments are allowed for subclasses of ``NearestNeighbors``.def__init__(self,*,a:int=0):# the __init__ parameters are injected as ``parameters``print(self.parameters.a)# 0# the parameters can be modified as neededself.parameters.a+=1print(self.parameters.a)# 1# a copy of the current global configuration is injected as ``config``print(self.config.save_compression)# 0# the configuration can be modified as needed (does not modify the global config)self.config.save_compression=1print(self.config.save_compression)# 1deffit(self,data:np.ndarray)->"Self":...defquery(self,point:np.ndarray,n_neighbors:int)->tuple[np.ndarray,np.ndarray]:...An interesting configuration aspect ismethods_require_fit, which specifies the set of methods that require a
successful call offitbefore they can be used. By default, the query methods are listed inmethods_require_fit, and, if a query method is called beforefit, an informative error message is shown.
A successful fit additionally sets theis_fittedproperty toTrueand removes the fit checks such that
there is zero overhead for queries. Manually settingis_fittedtoFalseagain adds the
checks to all methods specified inmethods_require_fit.