Package: oflibpytorch
Oflibpytorch:a handy pythonopticalflowlibrary, based onPyTorchtensors, that enables
the manipulation and combination of flow fields while keeping track of valid areas (see “Usage”) in the context of
machine learning algorithms implemented in PyTorch. It is mostly code written from scratch, but also contains useful
wrappers for specific functions from libraries such as PyTorch’sgrid_sample, to integrate them with the custom
flow field class introduced by oflibpytorch. Importantly, themain methods are differentiablewith respect to
their flow field tensor inputs, allowing for a seamless integration with machine learning algorithms. Features:All main methods that return a float tensor are differentiable with respect to tensor inputsA custom flow field class for both backward and forward (‘target’ / ‘source’ based) flow fields, handled as tensors
with a batch dimension, allowing for efficient batch-wise processing that can be performed on GPU if desiredA number of class methods to create flow fields from lists of affine transforms, or a transformation matrix, as
well as methods to resize the flow field, visualise it, warp images, or find necessary image padding, all while
keeping track of valid flow field areasA class method to process three different types of flow field combination operationsUtility functions including a PyTorch-based approximate interpolation to a grid from unstructured data as a
replacement for the SciPygriddatamethodOflibpytorch is based on oflibnumpy (code on GitHub,documentation on ReadTheDocs) and is aimed at allowing the
same operations to be performed with torch tensors instead of numpy arrays as far as currently feasible.Usage & DocumentationA user’s guide as well as full documentation of the library is available atReadTheDocs. Some quick examples:importoflibpytorchasofshape=(300,400)transform=[['rotation',200,150,-30]]# Make a flow field and display itflow=of.Flow.from_transforms(transform,shape)flow.show()# Alternative option without using the custom flow classflow=of.from_transforms(transform,shape,'t')of.show_flow(flow)Above:Visualisation of optical flow representing a rotation# Combine sequentially with another flow field, display the resultflow_2=of.Flow.from_transforms([['translation',40,0]],shape)result=of.combine_flows(flow,flow_2,mode=3)result.show(show_mask=True,show_mask_borders=True)# Alternative option without using the custom flow classflow_2=of.from_transforms([['translation',40,0]],shape,'t')result=of.combine_flows(flow,flow_2,mode=3,ref='t')of.show_flow(result)# Note: no way to show the valid flow area (see documentation)Above:Visualisation of optical flow representing a rotation, translated to the right, using the custom flow classresult.show_arrows(show_mask=True,show_mask_borders=True)# Alternative option without using the custom flow classof.show_flow_arrows(result,'t')# Note: again no way to show the valid flow areaAbove:Visualisation of optical flow representing a rotation, translated to the right, using the custom flow classInstallationIn order for oflibpytorch to work, the python environment needs to contain a PyTorch installation. To enable GPU usage,
the CUDA Toolkit is required as well. As it is difficult to guarantee an automatic installation via pip will use the
correct versions and work on all operating systems, it is left to the user to install PyTorch and Cudatoolkit
independently. The easiest route is a virtual conda environment and the recommended install command
from thePyTorch website, configured for the user’s specific system. To install oflibpytorch itself, use the
following command:pip install oflibpytorchTestingOflibpytorch contains a large number of tests to verify it is working as intended. Use the command line to navigate
tooflibpytorch/testand run the following code:python -m unittest discover .The tests will take several minutes to run. Successful completion will be marked withOK.Contribution & SupportSource Code:https://github.com/RViMLab/oflibpytorchIssue Tracker:https://github.com/RViMLab/oflibpytorch/issuesLicenseCopyright (c) 2022 Claudio S. Ravasio, PhD student at University College London (UCL), research assistant at King’s
College London (KCL), supervised by:Dr Christos Bergeles, PI of the Robotics and Vision in Medicine (RViM) lab in the School of Biomedical Engineering &
Imaging Sciences (BMEIS) at King’s College London (KCL)Prof Lyndon Da Cruz, consultant ophthalmic surgeon, Moorfields Eye Hospital, London UKThis code is licensed under theMIT License.If you use this code, please acknowledge us with the following citation:@article{ravasio_oflib,
  title     = {oflibnumpy {\&} oflibpytorch: Optical Flow Handling and Manipulation in Python},
  author    = {Ravasio, Claudio S. and Da Cruz, Lyndon and Bergeles, Christos},
  journal   = {Journal of Open Research Software (JORS)},
  year      = {2021},
  volume    = {9},
  publisher = {Ubiquity Press, Ltd.},
  doi       = {10.5334/jors.380}
}