Package: largesteps
Large Steps in Inverse Rendering of GeometryACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), December 2021.Baptiste Nicolet·Alec Jacobson·Wenzel JakobTable of ContentsInstallationParameterization package installationCloning the repositoryDownloading the scenesParameterizationRunning the experimentsRepository structureLicenseCitationAcknowledgmentsInstallationThis repository contains both the operators needed to use our parameterization
of vertex positions of meshes as well as the code for the experiments we show in
the paper.Parameterization package installationIf you are only interested in using our parameterization in an existing (PyTorch
based) pipeline, we have made it available to install viapip:pipinstalllargestepsThis will install thelargestepsmodule. This only contains the
parameterization logic implemented as a PyTorch custom operator. See thetutorialfor an example use case.Cloning the repositoryOtherwise, if you want to reproduce the experiments from the paper, you can
clone this repo and install the module locally.gitclone--recursivegit@github.com:rgl-epfl/large-steps-pytorch.gitcdlarge-steps-pytorch
pipinstall.The experiments in this repository depend on PyTorch. Please follow instructions on
the PyTorchwebsiteto install it.To installnvdiffrastand the Botsch-Kobbelt remesher, which are provided as
submodules, please run thesetup_dependencies.shscript.nvdiffrastrelies on thecudatoolkit-devpackage to compile modules at runtime.
To install it with Anaconda:condainstall-cconda-forgecudatoolkit-devTo install the other dependencies needed to run the experiments, also run:pipinstall-rrequirements.txt:warning: On Linux,nvdiffrastrequires usingg++to compile some PyTorch
extensions, make sure this is your default compiler:exportCC=gcc&&CXX=g++Rendering the figures will also require installingblender. You can specify the name of the
blender executable you wish to use inscripts/constants.pyDownloading the scenesThe scenes for the experiments can be downloadedhere.
Please extract the archive at the toplevel of this repository.ParameterizationIn a nutshell, our parameterization can be obtained in just a few lines:# Given tensors v and f containing vertex positions and facesfromlargesteps.geometryimportlaplacian_uniform,compute_matrixfromlargesteps.parameterizeimportto_differential,from_differential# Compute the system matrixM=compute_matrix(v,f,lambda_=10)# Parameterizeu=to_differential(M,v)compute_matrixreturns the parameterization matrixM=I+ λL.
This function takes another parameter,alpha, which leads to a slightly
different, but equivalent, formula for the matrix:M= (1-α)I+ αL,
with α ∈ [0,1[. With this formula, the scale of the matrixMhas the same
order of magnitude regardless of α.M=compute_matrix(L,alpha=0.9)Then, vertex coordinates can be retrieved as:v=from_differential(u,M,method='Cholesky')This will in practice perform a cache lookup for a solver associated to the
matrixM(and instantiate one if not found) and solve the linear systemMv=u. Further calls tofrom_differentialwith the same
matrix will use the solver stored in the cache. Since this operation is
implemented as a differentiable PyTorch operation, there is nothing more to be
done to optimize this parameterization.Running the experimentsYou can then run the experiments in thefiguresfolder, in which each
subfolder corresponds to a figure in the paper, and contains two files:generate_data.py: contains the script to run the experiment and write the
output to the directory specified inscripts/constants.pyfigure.ipynb: contains the script generating the figure, assuminggenerate_data.pyhas been run before and the output written to the directory
specified inscripts/constants.pyWe provide the scripts for the following figures:Fig. 1 ->teaserFig. 3 ->multiscaleFig. 5 ->remeshingFig. 6 ->reg_failFig. 7 ->comparisonFig. 8 ->viewpointsFig. 9 ->influence:warning: Several experiments are equal-time comparisons ran on a Linux Ryzen
3990X workstation with a TITAN RTX graphics card. In order to ensure
reproducibility, we have frozen the step counts for each method in these
experiments.Repository structureThelargestepsfolder contains the parameterization module made available viapip. It contains:geometry.py: contains the laplacian matrix computation.optimize.py: contains theAdamUniformoptimizer implementationparameterize.py: contains the actual parameterization code, implemented as ato_differentialandfrom_differentialfunction.solvers.py: contains the Cholesky and conjugate gradients solvers used to
convert parameterized coordinates back to vertex coordinates.Other functions used for the experiments are included in thescriptsfolder:blender_render.py: utility script to render meshes inside blenderconstants.py: contains paths to different useful folders (scenes, remesher, etc.)geometry.py: utility geometry functions (normals computation, edge length, etc.)io_ply.py: PLY mesh file loadingload_xml.py: XML scene file loadingmain.py: contains the main optimization functionpreamble.py: utility scipt to a import redundant modules for the figuresrender.py: contains the rendering logic, usingnvdiffrastLicenseThis code is provided under a 3-clause BSD license that can be found in the
LICENSE file. By using, distributing, or contributing to this project, you agree
to the terms and conditions of this license.CitationIf you use this code for academic research, please cite our method using the following BibTeX entry:@article{Nicolet2021Large,author="Nicolet, Baptiste and Jacobson, Alec and Jakob, Wenzel",title="Large Steps in Inverse Rendering of Geometry",journal="ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia)",volume="40",number="6",year="2021",month=dec,doi="10.1145/3478513.3480501",url="https://rgl.epfl.ch/publications/Nicolet2021Large"}AcknowledgmentsThe authors would like to thank Delio Vicini for early discussions about this
project, Silvia Sellán for sharing her remeshing implementation and help for the
figures, as well as Hsueh-Ti Derek Liu for his advice in making the figures.
Also, thanks to Miguel Crespo for making this README template.