Package: gpytorch-lattice-kernel
Simplex-GPsThis repository hosts the code forSKIing on Simplices: Kernel Interpolation on the Permutohedral Lattice for Scalable Gaussian Processes(Simplex-GPs) bySanyam Kapoor,Marc Finzi,Ke Alexander Wang,Andrew Gordon Wilson.The IdeaFast matrix-vector multiplies (MVMs) are the cornerstone of modern scalable
Gaussian processes. By building upon the approximation proposed byStructured Kernel Interpolation(SKI),
and leveraging advances in fast high-dimensional image filtering,
Simplex-GPs approximate the computation of the kernel matrices by tiling the
space using asparse permutohedral lattice, instead of a rectangular grid.The matrix-vector product implied by the kernel operations in SKI are now
approximated via the three stages visualized above ---splat(projection onto the permutohedral lattice),blur(applying the blur operation as a matrix-vector product), andslice(re-projecting back into the original space).This alleviates the curse of dimensionality associated with SKI operations,
allowing them to scale beyond ~5 dimensions, and provides competitive advantages
in terms of runtime and memory costs, at little expense of downstream performance.
See our manuscript for complete details.UsageThe lattice kernels are packaged as GPyTorch modules, and can be used as a
fast approximation to either theRBFKernelor theMaternKernel.
The corresponding replacement modules areRBFLatticeandMaternLattice.RBFLatticekernel is simple to use by changing a single line of code:import gpytorch as gp
from gpytorch_lattice_kernel import RBFLattice

class SimplexGPModel(gp.models.ExactGP):def __init__(self, train_x, train_y):likelihood = gp.likelihoods.GaussianLikelihood()super().__init__(train_x, train_y, likelihood)self.mean_module = gp.means.ConstantMean()self.covar_module = gp.kernels.ScaleKernel(-      gp.kernels.RBFKernel(ard_num_dims=train_x.size(-1))+      RBFLattice(ard_num_dims=train_x.size(-1), order=1))def forward(self, x):mean_x = self.mean_module(x)covar_x = self.covar_module(x)return gp.distributions.MultivariateNormal(mean_x, covar_x)TheGPyTorch Regression Tutorialprovides a simpler example on toy data, where this kernel can be used as a
drop-in replacement.InstallTo use the kernel in your code, install the package as:pipinstallgpytorch-lattice-kernelNOTE: The kernel is compiled lazily from source usingCMake.
If the compilation fails, you may need to install a more recent version.
Additionally,ninjais required for compilation. One way to install is:condainstall-cconda-forgecmakeninjaLocal SetupFor a local development setup, create thecondaenvironment$condaenvcreate-fenvironment.ymlRemember to add the root of the project to PYTHONPATH if not already.$exportPYTHONPATH="$(pwd):${PYTHONPATH}"TestTo verify the code is working as expected, a simpletest fileis provided, that tests for the training marginal likelihood achieved by
Simplex-GPs and Exact-GPs. Run as:pythontests/train_snelson.pyTheSnelson 1-D toy datasetis used.
A copy is available insnelson.csv.ResultsThe proposed kernel can be used with GPyTorch as usual. An example script to
reproduce results is,pythonexperiments/train_simplexgp.py--dataset=elevators--data-dir=<path/to/uci/data/mat/files>We useFireto handle CLI arguments.
All arguments of themainfunction are therefore valid arguments to the CLI.All figures in the paper can be reproduced vianotebooks.NOTE: The UCI datasetmatfiles are availablehere.LicenseApache 2.0