Package: geoopt
Manifold awarepytorch.optim.Unofficial implementation for“Riemannian Adaptive Optimization
Methods”ICLR2019 and more.InstallationMake sure you have pytorch>=1.10.2 installedThere are two ways to install geoopt:GitHub (preferred so far) due to active developmentpipinstallgit+https://github.com/geoopt/geoopt.gitpypi (this might be significantly behind master branch)pipinstallgeooptThe preferred way to install geoopt will change once stable project stage is achieved.
Now, pypi is behind master as we actively develop and implement new features.PyTorch SupportGeoopt officially supports2 latest stable versionsof pytorch upstream or the latest major release.What is done so farWork is in progress but you can already use this. Note that API might
change in future releases.Tensorsgeoopt.ManifoldTensor- just as torch.Tensor with additionalmanifoldkeyword argument.geoopt.ManifoldParameter- same as above, recognized intorch.nn.Module.parametersas correctly subclassed.All above containers have special methods to work with them as with
points on a certain manifold.proj_()- inplace projection on the manifold..proju(u)- project vectoruon the tangent space. You need
to project all vectors for all methods below..egrad2rgrad(u)- project gradientuon Riemannian manifold.inner(u, v=None)- inner product at this point for twotangentvectors at this point. The passed vectors are not
projected, they are assumed to be already projected..retr(u)- retraction map following vectoru.expmap(u)- exponential map following vectoru(if expmap is not available in closed form, best approximation is used).transp(v, u)- transport vectorvwith directionu.retr_transp(v, u)- transportself, vectorv(and possibly more vectors) with directionu(returns are plain tensors)Manifoldsgeoopt.Euclidean- unconstrained manifold inRwith
Euclidean metricgeoopt.Stiefel- Stiefel manifold on matricesA inR^{nx p} : A^t A=I,n >= pgeoopt.Sphere- Sphere manifold||x||=1geoopt.BirkhoffPolytope- manifold of Doubly Stochastic matricesgeoopt.Stereographic- Constant curvature stereographic projection modelgeoopt.SphereProjection- Sphere stereographic projection modelgeoopt.PoincareBall-Poincare ball modelgeoopt.Lorentz-Hyperboloid modelgeoopt.ProductManifold- Product manifold constructorgeoopt.Scaled- Scaled version of the manifold. Similar toLearning Mixed-Curvature Representations in Product Spacesif combined withProductManifoldgeoopt.SymmetricPositiveDefinite- SPD matrix manifoldgeoopt.UpperHalf- Siegel Upper half manifold. Supports Riemannian and Finsler metrics, as inSymmetric Spaces for Graph Embeddings: A Finsler-Riemannian Approach.geoopt.BoundedDomain- Siegel Bounded domain manifold. Supports Riemannian and Finsler metrics.All manifolds implement methods necessary to manipulate tensors on manifolds and
tangent vectors to be used in general purpose. See more indocumentation.Optimizersgeoopt.optim.RiemannianSGD- a subclass oftorch.optim.SGDwith the same APIgeoopt.optim.RiemannianAdam- a subclass oftorch.optim.AdamSamplersgeoopt.samplers.RSGLD- Riemannian Stochastic Gradient Langevin
Dynamicsgeoopt.samplers.RHMC- Riemannian Hamiltonian Monte-Carlogeoopt.samplers.SGRHMC- Stochastic Gradient Riemannian
Hamiltonian Monte-CarloLayersExperimentalgeoopt.layersmodule allows to embed geoopt into deep learningCiting GeooptIf you find this project useful in your research, please kindly add this bibtex entry in references and cite.@misc{geoopt2020kochurov,title={Geoopt: Riemannian Optimization in PyTorch},author={Max Kochurov and Rasul Karimov and Serge Kozlukov},year={2020},eprint={2005.02819},archivePrefix={arXiv},primaryClass={cs.CG}}DonationsETH: 0x008319973D4017414FdF5B3beF1369bA78275C6A