Package: scikit-hubness
scikit-hubnessscikit-hubnesscomprises tools for the analysis and
reduction of hubness in high-dimensional data.
Hubness is an aspect of thecurse of dimensionalityand is detrimental to many machine learning and data mining tasks.Theskhubness.analysisandskhubness.reductionpackages allow toanalyze, whether your data sets show hubnessreduce hubness via a variety of different techniquesperform downstream analysis (performance assessment) withscikit-learndue to compatible data structuresTheskhubness.neighborspackage acts as a drop-in replacement forsklearn.neighbors.
In addition to the functionality inherited fromscikit-learn,
it also featuresapproximate nearest neighborsearchhubness reductionand combinations,which allows for fast hubness-reduced neighbor search in large datasets
(tested with >1M objects).We follow the API conventions and code style ofscikit-learn.InstallationMake sure you have a working Python3 environment (at least 3.7).Use pip to install the latest stable version ofscikit-hubnessfrom PyPI:pipinstallscikit-hubnessDependencies are installed automatically, if necessary.scikit-hubnessrequiresnumpy,scipyandscikit-learn.
Approximate nearest neighbor search and approximate hubness reduction
additionally requires at least one of the following packges:nmslibfor hierachical navigable small-world graphs ('hnsw')ngtpyfor nearest neighbor graphs ('nng'), and variants (ANNG, ONNG)puffinnfor locality-sensitive hashing ('lsh')falconnfor alternative LSH ('falconn_lsh') , orannoyfor random projection forests ('rptree').Some modules requiretqdmorjoblib. All these packages are available
from open repositories, such asPyPI.For more details and alternatives, please see theInstallation instructions.DocumentationDocumentation is available online:http://scikit-hubness.readthedocs.io/en/latest/index.htmlWhat's newSee thechangelogto find what's new in the latest package version.QuickstartUsers ofscikit-hubnessmay want toanalyse, whether their data show hubnessreduce hubnessperform learning (classification, regression, ...)The following example shows all these steps for an example dataset
from the text domain (dexter). (Please make sure you have installedscikit-hubness).# load the example dataset 'dexter'fromskhubness.dataimportload_dexterX,y=load_dexter()# dexter is embedded in a high-dimensional space,# and could, thus, be prone to hubnessprint(f'X.shape ={X.shape}, y.shape={y.shape}')# assess the actual degree of hubness in dexterfromskhubnessimportHubnesshub=Hubness(k=10,metric='cosine')hub.fit(X)k_skew=hub.score()print(f'Skewness ={k_skew:.3f}')# additional hubness indices are available, for example:print(f'Robin hood index:{hub.robinhood_index:.3f}')print(f'Antihub occurrence:{hub.antihub_occurrence:.3f}')print(f'Hub occurrence:{hub.hub_occurrence:.3f}')# There is considerable hubness in dexter.# Let's see, whether hubness reduction can improve# kNN classification performancefromsklearn.model_selectionimportcross_val_scorefromskhubness.neighborsimportKNeighborsClassifier# vanilla kNNknn_standard=KNeighborsClassifier(n_neighbors=5,metric='cosine')acc_standard=cross_val_score(knn_standard,X,y,cv=5)# kNN with hubness reduction (mutual proximity)knn_mp=KNeighborsClassifier(n_neighbors=5,metric='cosine',hubness='mutual_proximity')acc_mp=cross_val_score(knn_mp,X,y,cv=5)print(f'Accuracy (vanilla kNN):{acc_standard.mean():.3f}')print(f'Accuracy (kNN with hubness reduction):{acc_mp.mean():.3f}')# Accuracy was considerably improved by mutual proximity.# Did it actually reduce hubness?hub_mp=Hubness(k=10,metric='cosine',hubness='mutual_proximity')hub_mp.fit(X)k_skew_mp=hub_mp.score()print(f'Skewness after MP:{k_skew_mp:.3f}'f'(reduction of{k_skew-k_skew_mp:.3f})')print(f'Robin hood:{hub_mp.robinhood_index:.3f}'f'(reduction of{hub.robinhood_index-hub_mp.robinhood_index:.3f})')# The neighbor graph can also be created directly,# with or without hubness reductionfromskhubness.neighborsimportkneighbors_graphneighbor_graph=kneighbors_graph(X,n_neighbors=5,hubness='mutual_proximity')Check theUser Guidefor additional example usage.DevelopmentThe developers ofscikit-hubnesswelcome all kinds of contributions!
Get in touch with us if you have comments,
would like to see an additional feature implemented,
would like to contribute code or have any other kind of issue.
Don't hesitate to file anissuehere on GitHub.For more information about contributing, please have a look at thecontributors guidelines.(c) 2018-2019, Roman Feldbauer
Austrian Research Institute for Artificial Intelligence (OFAI) and
University of Vienna, Division of Computational Systems Biology (CUBE)
Contact: <roman.feldbauer@univie.ac.at>CitationA software publication paper is currently in preparation. Until then,
if you usescikit-hubnessin your scientific publication, please cite:@INPROCEEDINGS{8588814,
author={R. {Feldbauer} and M. {Leodolter} and C. {Plant} and A. {Flexer}},
booktitle={2018 IEEE International Conference on Big Knowledge (ICBK)},
title={Fast Approximate Hubness Reduction for Large High-Dimensional Data},
year={2018},
volume={},
number={},
pages={358-367},
keywords={computational complexity;data analysis;data mining;mobile computing;public domain software;software packages;mobile device;open source software package;high-dimensional data mining;fast approximate hubness reduction;massive mobility data;linear complexity;quadratic algorithmic complexity;dimensionality curse;Complexity theory;Indexes;Estimation;Data mining;Approximation algorithms;Time measurement;curse of dimensionality;high-dimensional data mining;hubness;linear complexity;interpretability;smartphones;transport mode detection},
doi={10.1109/ICBK.2018.00055},
ISSN={},
month={Nov},}The technical reportFast approximate hubness reduction for large high-dimensional datais available atOFAI.Additional readingLocal and Global Scaling Reduce Hubs in Space, Journal of Machine Learning Research 2012,Link.A comprehensive empirical comparison of hubness reduction in high-dimensional spaces,
Knowledge and Information Systems 2018,DOI.Licensescikit-hubnessis licensed under the terms of the BSD-3-Clauselicense.
Theskhubness.neighborspackage was modified fromsklearn.neighbors,
distributed under the samelicense.
Users can, therefore, safely usescikit-hubnessin the same way they
usescikit-learn.Note:
Individual files contain the following tag instead of the full license text.SPDX-License-Identifier: BSD-3-ClauseThis enables machine processing of license information based on the SPDX
License Identifiers that are here available:https://spdx.org/licenses/AcknowledgementsSeveral parts ofscikit-hubnessadapt code fromscikit-learn.
We thank all the authors and contributors of this project
for the tremendous work they have done.PyVmMonitor is being used to support the development of this free open source
software package. For more information go tohttp://www.pyvmmonitor.com