Package: torchpairwise
TorchPairwiseThis package provides highly-efficient pairwise metrics forPyTorch.Newsv0.1.1:AddedSNRdistance (torchpairwise.snr_distances) presented inhttps://arxiv.org/abs/1904.02616.Highlightstorchpairwiseis a collection ofgeneral purposepairwise metric functions that behave similar totorch.cdist(which only implements $L_p$ distance).
Instead, we offer a lot more metrics ported from other packages such asscipy.spatial.distanceandsklearn.metrics.pairwise.
For task-specific metrics (e.g. for evaluation of classification, regression, clustering, ...), you should be in the
wrong place, please head to theTorchMetricsrepo.Written intorch's C++ API, the main differences are that our metrics:are all (except some boolean distances)differentiablewith backward formulas manually derived, implemented,
and verified withtorch.autograd.gradcheck.arebatchedand can exploit GPU parallelization.can be integrated seamlessly withinPyTorch-based projects, all functions aretorch.jit.script-able.List of pairwise distance metricstorchpairwiseopsEquivalences in other librariesDifferentiableeuclidean_distancessklearn.metrics.pairwise.euclidean_distances✔️haversine_distancessklearn.metrics.pairwise.haversine_distances✔️manhattan_distancessklearn.metrics.pairwise.manhattan_distances✔️cosine_distancessklearn.metrics.pairwise.cosine_distances✔️l1_distances(Alias ofmanhattan_distances)✔️l2_distances(Alias ofeuclidean_distances)✔️lp_distances(Alias ofminkowski_distances)✔️linf_distances(Alias ofchebyshev_distances)✔️directed_hausdorff_distancesscipy.spatial.distance.directed_hausdorff[^1]✔️minkowski_distancesscipy.spatial.distance.minkowski[^1]✔️wminkowski_distancesscipy.spatial.distance.wminkowski[^1]✔️sqeuclidean_distancesscipy.spatial.distance.sqeuclidean_distances[^1]✔️correlation_distancesscipy.spatial.distance.correlation[^1]✔️hamming_distancesscipy.spatial.distance.hamming[^1]❌[^2]jaccard_distancesscipy.spatial.distance.jaccard[^1]❌[^2]kulsinski_distancesscipy.spatial.distance.kulsinski[^1]❌[^2]kulczynski1_distancesscipy.spatial.distance.kulczynski1[^1]❌[^2]seuclidean_distancesscipy.spatial.distance.seuclidean[^1]✔️cityblock_distancesscipy.spatial.distance.cityblock[^1](Alias ofmanhattan_distances)✔️mahalanobis_distancesscipy.spatial.distance.mahalanobis[^1]✔️chebyshev_distancesscipy.spatial.distance.chebyshev[^1]✔️braycurtis_distancesscipy.spatial.distance.braycurtis[^1]✔️canberra_distancesscipy.spatial.distance.canberra[^1]✔️jensenshannon_distancesscipy.spatial.distance.jensenshannon[^1]✔️yule_distancesscipy.spatial.distance.yule[^1]❌[^2]dice_distancesscipy.spatial.distance.dice[^1]❌[^2]rogerstanimoto_distancesscipy.spatial.distance.rogerstanimoto[^1]❌[^2]russellrao_distancesscipy.spatial.distance.russellrao[^1]❌[^2]sokalmichener_distancesscipy.spatial.distance.sokalmichener[^1]❌[^2]sokalsneath_distancesscipy.spatial.distance.sokalsneath[^1]❌[^2]snr_distancespytorch_metric_learning.distances.SNRDistance[^1]✔️[^1]: These metrics are not pairwise but a pairwise form can be computed by
callingscipy.spatial.distance.cdist(x1, x2, metric="[metric_name_or_callable]").[^2]: These are boolean distances.hamming_distancescan be applied for floating point inputs but involves
comparison.Other pairwise metrics or kernel functionsThese metrics are usually used to compute kernel for machine learning algorithms.torchpairwiseopsEquivalences in other librariesDifferentiablelinear_kernelsklearn.metrics.pairwise.linear_kernel✔️polynomial_kernelsklearn.metrics.pairwise.polynomial_kernel✔️sigmoid_kernelsklearn.metrics.pairwise.sigmoid_kernel✔️rbf_kernelsklearn.metrics.pairwise.rbf_kernel✔️laplacian_kernelsklearn.metrics.pairwise.laplacian_kernel✔️cosine_similaritysklearn.metrics.pairwise.cosine_similarity✔️additive_chi2_kernelsklearn.metrics.pairwise.additive_chi2_kernel✔️chi2_kernelsklearn.metrics.pairwise.chi2_kernel✔️CustomcdistandpdistFurthermore, we provide a convenient wrapper function analoguous totorch.cdistexcepts that it takes a stringmetric: str = "minkowski"indicating the desired metric to be used as the third argument,
and extra metric-specific arguments are passed as keywords.importtorch,torchpairwise# directed_hausdorff_distances is a pairwise 2d metricx1=torch.rand(10,6,3)x2=torch.rand(8,5,3)generator=torch.Generator().manual_seed(1)output=torchpairwise.cdist(x1,x2,metric="directed_hausdorff",shuffle=True,# kwargs exclusive to directed_hausdorffgenerator=generator)Note that pairwise metrics on the second table are currently not allowed keys forcdistbecause they are notdist.
We have a similar plan forpdist(which is equivalent to callingcdist(x1, x1)but avoid storing duplicated
positions).
However, that requires a total overhaul of existing C++/Cuda kernels and won't be available soon.Future ImprovementsAdd more metrics (contact me or create a feature request issue).Add memory-efficientargkminfor retrieving pairwise neighbors' distances and indices without storing the whole
pairwise distance matrix.Add an equivalence oftorch.pdistwithmetric: str = "minkowski"argument.(Unlikely) Support sparse layouts.Requirementstorch>=2.1.0(torch>=1.9.0if compiled from source)InstallationFrom PyPI:To install prebuilt wheels fromtorchpairwise, simply run:pip install torchpairwiseNote that the Linux and Windows wheels inPyPIare compiled withtorch==2.1.0andCuda 12.1.
We only do a non-strict version checking and a warning will be raised iftorch's andtorchpairwise's
Cuda versions do not match.From Source:Make sure your machine has a C++17 and a Cuda compiler installed, then clone the repo and run:pip install .UsageThe basic usecase is very straight-forward if you are familiar withsklearn.metrics.pairwiseandscipy.spatial.distance:scikit-learn / SciPyTorchPairwiseimportnumpyasnpimportsklearn.metrics.pairwiseassklearn_pairwisex1=np.random.rand(10,5)x2=np.random.rand(12,5)output=sklearn_pairwise.cosine_similarity(x1,x2)print(output)importtorchimporttorchpairwisex1=torch.rand(10,5,device='cuda')x2=torch.rand(12,5,device='cuda')output=torchpairwise.cosine_similarity(x1,x2)print(output)importnumpyasnpimportscipy.spatial.distanceasdistancex1=np.random.binomial(1,p=0.6,size=(10,5)).astype(np.bool_)x2=np.random.binomial(1,p=0.7,size=(12,5)).astype(np.bool_)output=distance.cdist(x1,x2,metric='jaccard')print(output)importtorchimporttorchpairwisex1=torch.bernoulli(torch.full((10,5),fill_value=0.6,device='cuda')).to(torch.bool)x2=torch.bernoulli(torch.full((12,5),fill_value=0.7,device='cuda')).to(torch.bool)output=torchpairwise.jaccard_distances(x1,x2)print(output)Please check thetestsfolder where we will add more examples.LicenseThe code is released under the MIT license. SeeLICENSE.txtfor details.