Package: pyequilib
equilibProcessing Equirectangular Images with PythonA library for processing equirectangular image that runs on Python.Developed using Python>=3.6 (c++is WIP).Compatible withcudatensors for faster processing.No need for other dependencies except fornumpyandtorch.Added functionality like creating rotation matrices, batched processing, and automatic type detection.Highly modularIf you found this module helpful to your project, please site this repository:@software{pyequilib2021github,
  author = {Haruya Ishikawa},
  title = {PyEquilib: Processing Equirectangular Images with Python},
  url = {http://github.com/haruishi43/equilib},
  version = {0.5.0},
  year = {2021},
}Installation:Prerequisites:Python (>=3.6)Pytorch (tested on 1.12)pipinstallpyequilibFor developing, use:gitclone--recursivehttps://github.com/haruishi43/equilib.gitcdequilib

pipinstall-rrequirements.txt

pipinstall-e.# orpythonsetup.pydevelopNOTE: might not work for PyTorch>=2.0. If you have any issues, please open an issue.Basic Usage:equilibhas different transforms of equirectangular (or cubemap) images (note each transform hasclassandfuncAPIs):Cube2Equi/cube2equi: cubemap to equirectangular transformEqui2Cube/equi2cube: equirectangular to cubemap transformEqui2Equi/equi2equi: equirectangular transformEqui2Pers/equi2pers: equirectangular to perspective transformThere are norealdifferences inclassorfuncAPIs:classAPIs will allow instantiating a class which you can call many times without having to specify configurations (classAPIs call thefuncAPI)funcAPIs are useful when there are no repetitive callsbothclassandfuncAPIs are extensible, so you can extend them to your use-cases or create a method that's more optimized (pull requests are welcome btw)Each API automatically detects the input type (numpy.ndarrayortorch.Tensor), and outputs are the same type.An example forEqui2Pers/equi2pers:Equi2Persequi2persimportnumpyasnpfromPILimportImagefromequilibimportEqui2Pers# Input equirectangular imageequi_img=Image.open("./some_image.jpg")equi_img=np.asarray(equi_img)equi_img=np.transpose(equi_img,(2,0,1))# rotationsrots={'roll':0.,'pitch':np.pi/4,# rotate vertical'yaw':np.pi/4,# rotate horizontal}# Intialize equi2persequi2pers=Equi2Pers(height=480,width=640,fov_x=90.0,mode="bilinear",)# obtain perspective imagepers_img=equi2pers(equi=equi_img,rots=rots,)importnumpyasnpfromPILimportImagefromequilibimportequi2pers# Input equirectangular imageequi_img=Image.open("./some_image.jpg")equi_img=np.asarray(equi_img)equi_img=np.transpose(equi_img,(2,0,1))# rotationsrots={'roll':0.,'pitch':np.pi/4,# rotate vertical'yaw':np.pi/4,# rotate horizontal}# Run equi2perspers_img=equi2pers(equi=equi_img,rots=rots,height=480,width=640,fov_x=90.0,mode="bilinear",)For more information about how each APIs work, take a look in.readmeor go through example codes in thetestsorscripts.Coordinate System:Right-handed rule XYZ global coordinate system.x-axisfaces forward andz-axisfaces up.roll: counter-clockwise rotation about thex-axispitch: counter-clockwise rotation about they-axisyaw: counter-clockwise rotation about thez-axisYou can chnage the right-handed coordinate system so that thez-axisfaces down by addingz_down=Trueas a parameter.See demo scripts underscripts.Grid SamplingTo process equirectangular images fast, whether to crop perspective images from the equirectangular image, the library takes advantage of grid sampling techniques.
Some sampling techniques are already implemented, such asscipy.ndimage.map_coordiantesandcv2.remap.
This project's goal was to reduce these dependencies and usecudaand batch processing withtorchandc++for a faster processing of equirectangular images.
There were not many projects online for these purposes.
In this library, we implement varieties of methods usingc++,numpy, andtorch.
This part of the code needscudaacceleration because grid sampling is parallelizable.
Fortorch, the built-intorch.nn.functional.grid_samplefunction is very fast and reliable.
I have implemented apuretorchimplementation ofgrid_samplewhich is very customizable (might not be fast as the native function).
Fornumpy, I have implemented grid sampling methods that are faster thanscipyand more robust thancv2.remap.
Just like with this implementation oftorch,numpyimplementation is just as customizable.
It is also possible to pass thescipyandcv2's grid sampling function through the use ofoverride_funcargument ingrid_sample.
Developingfasterapproaches andc++methods areWIP.
Seeherefor more info on implementations.Some notes:By default,numpy'sgrid_samplewill use purenumpyimplementation. It is possible to override this implementation withscipyandcv2's implementation usingoverride_func.By default,torch'sgrid_samplewill use the official implementation.Benchmarking codes are stored intests/. For example, benchmarking codes fornumpy'sequi2persis located intests/equi2pers/numpy_run_baselines.pyand you can benchmark the runtime performance using different parameters againstscipyandcv2.Develop:Test files forequilibare included undertests.Running tests:pytesttestsNote that I have added codes to benchmark every step of the process so that it is possible to optimize the code.
If you find there are optimal ways of the implementation or bugs, all pull requests and issues are welcome.CheckCONTRIBUTING.mdfor more informationTODO:Documentations for each transformAdd table and statistics for speed improvementsBatch processing fornumpyMixed precision fortorchc++version of grid samplingMore accurate intrinsic matrix formulation using vertial FOV forequi2persMultiprocessing support (slow when running ontorch.distributed)Acknowledgements:py360convertPerspective-and-Equirectangular