Package: deltaconv
DeltaConv[Paper][Project page]Code for the SIGGRAPH 2022 paper "DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds" by Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, and Klaus Hildebrandt.Anisotropic convolution is a central building block of CNNs but challenging to transfer to surfaces. DeltaConv learns combinations and compositions of operators from vector calculus, which are a natural fit for curved surfaces. The result is a simple and robust anisotropic convolution operator for point clouds with state-of-the-art results.Top: unlike images, surfaces have no global coordinate system. Bottom: DeltaConv learns both scalar and vector features using geometric operators.ContentsInstallationReplicating the experimentsTestsCitationInstallationClone this repository:gitclonehttps://github.com/rubenwiersma/deltaconv.gitCreate a conda environment from theenvironment.yml:condaenvcreate-ndeltaconv-fenvironment.ymlDone!Manual installationIf you wish to install DeltaConv in your own environment, proceed as follows.Make sure that you have installed:Numpy -pip install numpyPyTorch- seeinstructionsPyG-conda install pyg -c pygInstall DeltaConv:pipinstalldeltaconvBuilding DeltaConv for yourselfMake sure you clone the repository with submodules:gitclone--recurse-submoduleshttps://github.com/rubenwiersma/deltaconv.gitIf you have already cloned the repository without submodules, you can fix it withgit submodule update --init --recursive.Install from folder:cd[root_folder]pipinstallReplicating the experimentsSee the README.md inreplication_scriptsfor instructions on replicating the experiments and using the pre-trained weights (available inexperiments/pretrained_weights).In short, you can run bash scripts to replicate our experiments. For example, evaluating pre-trained weights on ShapeNet:cd[root_folder]condaactivatedeltaconv
bashreplication_scripts/pretrained/shapenet.shYou can also directly run the python files inexperiments:pythonexperiments/train_shapenet.pyUse the-hor--helpflag to find out which arguments can be passed to the training script:pythonexperiments/train_shapenet.py-hYou can keep track of the training process with tensorboard:tensorboardlogdir=experiments/runs/shapenet_allAnisotropic DiffusionThe code that was used to generate Figure 2 from the paper and Figure 2 and 3 from the supplement is a notebook in the folderexperiments/anisotropic_diffusion.DataThe training scripts assume that you have adatafolder inexperiments. ModelNet40 and ShapeNet download the datasets from a public repository. Instructions to download the data for human body shape segmentation, SHREC, and ScanObjectNN are given in the training scripts.TestsIn the paper, we make statements about a number of properties of DeltaConv that are either a result of prior work or due to the implementation. We created a test suite to ensure that these properties hold for the implementation, along with unit tests for each module. For example:Section 3.6, 3.7: Vector MLPs are equivariant to norm-preserving transformations, or coordinate-independent (rotations, reflections)test/nn/test_mlp.pytest/nn/test_nonlin.pySection 3.7: DeltaConv is coordinate-independent, a forward pass on a shape with one choice of bases leads to the same output and weight updates when run with different basestest/nn/test_deltaconv.pyIntroduction, section 3.2: The operators are robust to noise and outliers.test/geometry/test_grad_div.pySupplement, section 1: Vectors can be mapped between points with equation (15).test/geometry/test_grad_div.pyCitationsPlease cite our paper if this code contributes to an academic publication:@Article{Wiersma2022DeltaConv,author={Ruben Wiersma, Ahmad Nasikun, Elmar Eisemann, Klaus Hildebrandt},journal={Transactions on Graphics},title={DeltaConv: Anisotropic Operators for Geometric Deep Learning on Point Clouds},year={2022},month=jul,number={4},volume={41},doi={10.1145/3528223.3530166},publisher={ACM},}The farthest point sampling code relies on Geometry Central:@misc{geometrycentral,title={geometry-central},author={Nicholas Sharp and Keenan Crane and others},note={www.geometry-central.net},year={2019}}And we make use of PyG (and underlying packages) to load point clouds, compute sparse matrix products, and compute nearest neighbors:@inproceedings{Fey/Lenssen/2019,title={Fast Graph Representation Learning with {PyTorch Geometric}},author={Fey, Matthias and Lenssen, Jan E.},booktitle={ICLR Workshop on Representation Learning on Graphs and Manifolds},year={2019},}