Package: kiez
kiezA Python library for hubness reduced nearest neighbor search for the task of entity alignment with knowledge graph embeddings. The term kiez is agerman wordthat refers to a city neighborhood.Hubness ReductionHubness is a phenomenon that arises in high-dimensional data and describes the fact that a couple of entities are nearest neighbors (NN) of many other entities, while a lot of entities are NN to no one.
For entity alignment with knowledge graph embeddings we rely on NN search. Hubness therefore is detrimental to our matching results.
This library is intended to make hubness reduction techniques available to data integration projects that rely on (knowledge graph) embeddings in their alignment process. Furthermore kiez incorporates several approximate nearest neighbor (ANN) libraries, to pair the speed advantage of approximate neighbor search with increased accuracy of hubness reduction.InstallationYou can install kiez via pip:pipinstallkiezIf you have a GPU you can make kiez faster by installingfaiss(if you do not already have it in your environment):condaenvcreate-nkiez-faisspython=3.10
condaactivatekiez-faiss
condainstall-cpytorch-cnvidiafaiss-gpu=1.7.4mkl=2021blas=1.0=mkl
pipinstallkiezFor more information see theirinstallation instructions.You can also get other specific libraries with e.g.:pipinstallkiez[nmslib]UsageSimple nearest neighbor search for source entities in target space:fromkiezimportKiezimportnumpyasnp# create example datarng=np.random.RandomState(0)source=rng.rand(100,50)target=rng.rand(100,50)# fit and get neighborsk_inst=Kiez()k_inst.fit(source,target)nn_dist,nn_ind=k_inst.kneighbors()Using (A)NN libraries and hubness reduction methods:fromkiezimportKiezimportnumpyasnp# create example datarng=np.random.RandomState(0)source=rng.rand(100,50)target=rng.rand(100,50)# prepare algorithm and hubness reductionalgo_kwargs={"n_candidates":10}k_inst=Kiez(n_neighbors=5,algorithm="Faiss"algorithm_kwargs=algo_kwargs,hubness="CSLS")# fit and get neighborsk_inst.fit(source,target)nn_dist,nn_ind=k_inst.kneighbors()Torch SupportBeginning with version 0.5.0 torch can be used, when usingFaissas NN library:fromkiezimportKiezimporttorchsource=torch.randn((100,10))target=torch.randn((200,10))k_inst=Kiez(algorithm="Faiss",hubness="CSLS")k_inst.fit(source,target)nn_dist,nn_ind=k_inst.kneighbors()You can also utilize tensor on the GPU:k_inst=Kiez(algorithm="Faiss",algorithm_kwargs={"use_gpu":True},hubness="CSLS")k_inst.fit(source.cuda(),target.cuda())nn_dist,nn_ind=k_inst.kneighbors()DocumentationYou can find more documentation onreadthedocsBenchmarkThe results and configurations of our experiments can be found in a seperatebenchmarking repositoryCitationIf you find this work useful you can use the following citation:@article{obraczka2022fast,
  title={Fast Hubness-Reduced Nearest Neighbor Search for Entity Alignment in Knowledge Graphs},
  author={Obraczka, Daniel and Rahm, Erhard},
  journal={SN Computer Science},
  volume={3},
  number={6},
  pages={1--19},
  year={2022},
  publisher={Springer},
  url={https://link.springer.com/article/10.1007/s42979-022-01417-1},
  doi={10.1007/s42979-022-01417-1},
}ContributingPRs and enhancement ideas are always welcome. If you want to build kiez locally use:gitclonegit@github.com:dobraczka/kiez.gitcdkiez
poetryinstallTo run the tests (given you are in the kiez folder):poetryrunpytesttestsOr installnoxand run:noxwhich checks all the linting as well.Licensekiezis licensed under the terms of the BSD-3-Clauselicense.
Several files were modified fromscikit-hubness,
distributed under the samelicense.
The respective files contain the following tag instead of the full license text.SPDX-License-Identifier: BSD-3-ClauseThis enables machine processing of license information based on the SPDX
License Identifiers that are here available:https://spdx.org/licenses/