Package: hyperlib
HyperLib: Deep learning in the Hyperbolic spaceBackgroundThis library implements common Neural Network components in the hyperbolic space (using the Poincare model). The implementation of this library uses Tensorflow as a backend and can easily be used with Keras and is meant to help Data Scientists, Machine Learning Engineers, Researchers and others to implement hyperbolic neural networks.You can also use this library for uses other than neural networks by using the mathematical functions available in the Poincare class. In the future we may implement components that can be used in models other than neural networks. You can learn more about Hyperbolic networkshere, and in the references[^1] [^2] [^3] [^4].InstallThe recommended way to install is with pippip install hyperlibTo build from source, you need to compile the pybind11 extensions.For example to build on linux:conda-nhyperlibpython=3.8gxx_linux-64pybind11
pythonsetup.pyinstallHyperlib works with python>=3.8 and tensorflow>=2.0.Example UsageCreating a hyperbolic neural network using Keras:importtensorflowastffromtensorflowimportkerasfromhyperlib.nn.layers.lin_hypimportLinearHyperbolicfromhyperlib.nn.optimizers.rsgdimportRSGDfromhyperlib.manifold.poincareimportPoincare# Create layershyperbolic_layer_1=LinearHyperbolic(32,Poincare(),1)hyperbolic_layer_2=LinearHyperbolic(32,Poincare(),1)output_layer=LinearHyperbolic(10,Poincare(),1)# Create optimizeroptimizer=RSGD(learning_rate=0.1)# Create model architecturemodel=tf.keras.models.Sequential([hyperbolic_layer_1,hyperbolic_layer_2,output_layer])# Compile the model with the Riemannian optimizermodel.compile(optimizer=optimizer,loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],)Using math functions on the Poincare ball:importtensorflowastffromhyperlib.manifold.poincareimportPoincarep=Poincare()# Create two matricesa=tf.constant([[5.0,9.4,3.0],[2.0,5.2,8.9],[4.0,7.2,8.9]])b=tf.constant([[4.8,1.0,2.3]])# Matrix multiplication on the Poincare ballcurvature=1p.mobius_matvec(a,b,curvature)EmbeddingsA big advantage of hyperbolic space is its ability to represent hierarchical data. There are several techniques for embedding data in hyperbolic space; the most common is gradient methods [^6].If your data has a natural metric you can also use TreeRep[^5].
Input a symmetric distance matrix, or a compressed distance matriximportnumpyasnpfromhyperlib.embedding.treerepimporttreerepfromhyperlib.embedding.sarkarimportsarkar_embedding# Example: immunological distances between 8 mammals by Sarichcompressed_metric=np.array([32.,48.,51.,50.,48.,98.,148.,26.,34.,29.,33.,84.,136.,42.,44.,44.,92.,152.,44.,38.,86.,142.,42.,89.,142.,90.,142.,148.])# outputs a weighted networkx Graphtree=treerep(compressed_metric,return_networkx=True)# embed the tree in 2D hyperbolic spaceroot=0embedding=sarkar_embedding(tree,root,tau=0.5)Please see theexamples directoryfor complete examples.References[^1]:Chami, I., Ying, R., Ré, C. and Leskovec, J. Hyperbolic Graph Convolutional Neural Networks. NIPS 2019.[^2]:Nickel, M. and Kiela, D. Poincaré embeddings for learning hierarchical representations. NIPS 2017.[^3]:Khrulkov, Mirvakhabova, Ustinova, Oseledets, Lempitsky. Hyperbolic Image Embeddings.[^4]:Wei Peng, Varanka, Mostafa, Shi, Zhao. Hyperbolic Deep Neural Networks: A Survey.[^5]:Rishi Sonthalia and Anna Gilbert. Tree! I am no Tree! I am a Low Dimensional Hyperbolic Embedding[^6]:De Sa et. al. Representation Tradeoffs for Hyperbolic Embeddings