GPS: A Probabilistic Distributional Similarity with Gumbel Priors for Set-to-Set Matching
Authors: Ziming Zhang
Fangzhou Lin
Haotian Liu
Jose Morales
Haichong Zhang
Kazunori Yamada
Vijaya B Kolachalama
Venkatesh Saligrama
Venue: ICLR 2025 Poster (2025)
Keywords: Set-to-set matching
Gumbel prior distributions
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Set-to-set matching aims to identify correspondences between two sets of unordered items by minimizing a distance metric or maximizing a similarity measure. Traditional metrics, such as Chamfer Distance (CD) and Earth Mover’s Distance (EMD), are widely used for this purpose but often suffer from limitations like suboptimal performance in terms of accuracy and robustness, or high computational costs - or both. In this paper, we propose a novel, simple yet effective set-to-set matching similarity measure, GPS, based on Gumbel prior distributions. These distributions are typically used to model the extrema of samples drawn from various distributions. Our approach is motivated by the observation that the distributions of minimum distances from CD, as encountered in real world applications such as point cloud completion, can be accurately modeled using Gumbel distributions. We validate our method on tasks like few-shot image classification and 3D point cloud completion, demonstrating significant improvements over state of-the-art loss functions across several benchmark datasets. Our demo code is publicly available at https://github.com/Zhang-VISLab/ICLR2025-GPS

Published as a conference paper at ICLR 2025
GPS: A PROBABILISTIC DISTRIBUTIONAL SIMILARITY WITH GUMBEL PRIORS FOR SET-TO-SET MATCHING
Ziming Zhang': Fangzhou Lin't Haotian Liu'! Jose Morales', Haichong Zhang', Kazunori Yamada', Vijaya B Kolachalama', Venkatesh Saligrama*
Worcester Polytechnic Institute, USA Tohhok University, Japan Bossonn University, USA (zzhang15, flin2, hliu8, jamorales, hzhang10 y @ewpi.edu whok.attchok..ac. (vkola, srv) edu
ABSTRACT
Set-to-set matching aims to identify correspondences between two sets un- ordered items by minimizing a distance metric or maximizing similarity measure. Traditional metrics, such as Chamfer Distance (CD) and Earth Mover's 's Distance (EMD), widely used for this purpose but often suffer from limitations like suboptimal performance in terms of accuracy and robustness, or high computa- tional costs both. In this paper, propose an effective set-to-set matching similarity measure, GPS, based on Gumbel prior distributions, These distributions are typically used to model the extrema of samples drawn from various distri- butions. Our approach is motivated by the observation that the distributions of minimum distances from CD, as encountered in real-world applications such as point cloud completion, can be accurately modeled using Gumbel distributions. We validate our method on tasks like few-shot image classification and 3D point cloud completion, demonstrating significant improvements over state-of-the-ar loss func- tions across several benchmark datasets. Our demo code is publicly available at cwm/2hang 2hanghaaggggg Shang-VISLab. VISLVb SiiiaaaIICR22225--1 ptttps//ggite com/
INTRODUCTION
Problem. Sct-to-set matching involves comparing and identifying correspondences between two sets of items, which can be modeled as a bipartite graph matching problem. In this framework, the items in each set are represented as nodes on opposite sides of bipartite graph, with the edges representing the correspondences. This task is challenging due to several factors: (1) The matching process must be invariant to the order of both the sets and the items within them: (2) Finding effective representations for each set is critical, as it greatly influences performance; (3) Computing a meaningful similarity score between sets, often used as a loss function for learning feature extractors, is nontrivial problem. Our goal is to develop similarity measure that is both effective and efficient for set-to-set matching.
Distance Metrics. Set-to-set matching has been extensively studied in various research fields (Chang et al. [007]] Zhou et al. 2017a Saito et al.] [020) Jurewicz and Derczynski 2021 Yu et al. 202 a Kimura et al.) (023). Several distance metrics, such as Chamfer Distance (CD) (Yang et al. 2020),
Earth Mover's Distance (EMD) (Zhang et al.) [220aa) Yang et al.) 2024), and Wasserstein Distance (WD) (Zhu and Koniusz 2022), are commonly used to evaluate set-matching scores in different applications. These metrics can generally be considered special cases of optimal transportation (OT) for graph matching (Saad-Eldin et al. (021). which aims to find the most efficient way to move distribution of "materiais" another distribution of "consumers" by minimizing the total transportation cost. However, it is well known that these distance metrics often face challenges such as poor performance (e.g., matching accuracy and robustness of CD (Lin et al. (202aa) or high
computational complexity (e.g., EMD and WD (Nguyen et al. 2021 Rowland et al. 2019 Kolouri
First co-author, project leader
First co-author, responsible for experiments on point cloud completion
First co-author, responsible for experiments few-shot classification

---

Published as a conference paper at ICLR 2025
et al. 2019)), both, which limit their applicability in large-scale learning. To address this issue, some new distance metrics, such as density-aware CD (DCD) (Wu et al. 2021). HyperCD (Lin et al.) 2023b) and InfoCD (Lin et al.) 2023a), have been developed training losses in the literature to improve matching performance while achieving linear complexity similar to CD.
Limitations of Traditional Distance Metrics. Existing metrics fail to adequately capture the similarity between sets terms of their underlying distributions. In many real-world applications, such as classification,, the primary concern is not the direct similarity between individual data instances but rather their similarity in some (latent) space, such as class labels.
For instance, in Figure T in the context of Chamfer Dis- tance (CD), the distance between the sets and x is larger than that between the sets and +, despite o and x being sampled from the same underlying distri- bution. This highlights a critical issue: when asking "how likely are two sets of points to come from the same distribution?" CD and similar metrics may fail re- flect distributional similarity accurately. Consequently, traditional distance metrics are effective at measuring differences in observations (e.g., point clouds or images) but may struggle to capture deeper, potentially unknown abstractions such as distributions class labels.
0.5
-0.5
from Distr. 1
-0.5
0.5
Figure I: Illustration of three point sets ran- domly sampled from a circular distribution (i.e., Distr. 1) and Gaussian whose mean is conditional on the circle (i.e., Distr. 2).
Distributional Similarity. This concept widely ap- plied in natural language processing to assess word similarities based on the contexts in which they appear (Lee 2000). For example, to measure the similarity between words u and v, both words can be represented in vector space by counting their co-occurrences with words from predefined vocabulary within local contexts such as sentences or paragraphs. Various similarity functions are then applied these vectors compute their similarity. To adapt this idea for set-to-set matching, we treat the K nearest neighbors (KNNs) of a point from the opposite set as its "co-occurrences"
Gumbel Distributions. In probability theory, Gumbel distributions are used to model the distribution of ex- treme values (maxima minima) from various samples. We find that Gumbel distributions effectively model the KNN distances between two sets of feature vectors. This is shown in Figure 2 where the two 2D point sets are sampled from the circular distribution shown in Fig-
0.05
Daral NN
Data: Srd
0.05
:umbe: 2nd NN
Dumbe 1ist MN
Pobabbitty
0.04
Gumber 3hd
0.03
0.02
ure I We compute the negative-log distances between each point in one set and its 1st, 2nd, and 3rd nearest neighbors in the other set, transforming these distances into normalized histograms representing probabilities. These probability distributions are then fitted with Gum- bel distributions (see Definition]
10
Negative Log Distance
Figure 2: Data fitting with Gumbel distri- butions for 1st, 2nd, 3rd smallest distance distributions between two 2D point sets.
Our Approach and Contributions. We propose novel probabilistic method for set-to-set matching called GPS (Gumbel Prior Similarity) based on distributional similarity and Gumbel distributions. This method measures the similarity between the underlying distributions generating the sets. Specif- ically, we use the log- likellooooo of Gumbel distributions as our similarity measure, modeling the distributions of negative-log distances between the KNNs of the sets. GPS can be seamlessly inte- grated into existing neural network training frameworks, for instance, by using the negative of GPS loss function, while maintaining the same linear computational complexity as CD. We propose comprehensive analysis of GPS and its influence on learning behavior. We are the first to leverage statistical information from KNNs for set-to-set matching. To demonstrate the efficacy and efficiency of GPS, we conduct extensive experiments on tasks such as few-shot image classification and 3D point cloud completion, achieving state-of-tt oftie-----e--rt- performance across several benchmark datasets.
2

---

Published as a conference paper at ICLR 2025
2 RELATED WORK
Set-to-Set Matching, Many tasks in computer vision and machine learning, such as multiple instance learning (Isse et al., 22188] Maron and Lozano-Pérez, 1997), shape recognition (Su et al. 2015 Shi et al. 2015). and few-shot image classification (Afrasiyabi et al. (022), can be framed as set-to-set matching problems, where the goal is to identity correspondences between sets of instances. The unordered nature of these sets requires the extraction of invariant features that are not affected by the sequence of elements (Choi et al. 2018). Some research addresses this challenge by modifying neural network architectures. For example, Vinyals et al. (2016) introduced matching networks for one-shot learning, while Lee et al. (2019) proposed the Set Transformer, which uses an attention- based mechanism to model interactions among elements in input sets. [Saito et al. (2020) developed exchangeable deep neural networks. Recently, Kimura (2022) analyzed the generalization bounds for set-to-set matching with neural networks. The loss function for training these networks must also maintain order-invariance to effectively calculate distance functions between pairs of instances within the sets, as seen in DeepEMD (Zhang et al., (2020a).
Similarity (Metric) Learning. This research focuses on developing functions that measure the correlation between two objects (Ma and Manjunath| 1996, Balcan and Blum 2006 Yu et al. [0088] and has been successfully applied to various applications, including face recognition (Faraki et al.
2021 Cao al. (2013), few-shot image classification (Zhang et al., 2020a Oh et al. (022), emotion
matching (Lin et al. 20166), and re-identification (Zhou er 20.] Liao et al., 2017). Recently, these techniques have been integrated into deep learning (Liu et al. 2019 Cheng et al.. 20188) Ma
itt al. 2022 Liao et al. 2017: Zhou et al. 2017b) for representation learning in embedding spaces, where objects from the same set are closer together, and objects from different sets are further apart. However, learning a model from all sample pairs is challenging due to high computational complexity and poor local minima during training (Kaya and Bilge| 2019 Qian et al., 2019 Huang et al., 2016). Thus, designing effective and efficient loss functions a key issue in deep similarity learning (Elezi (et al. 2020. Wang et al. 2019). In contrast, we address this problem by utilizing the statistics of minimum distances between the items of sets to improve computational efficiency and performance.
Few-Shot Classification. This task aims to train a classifier to recognize both seen and unseen classes with limited labeled examples (Chen et al. (019a). During training, the model learns a generalized classification ability in varying classes (Oreshkin et al. 2018 Finn et al.] 2017). In the testing phase, when presented with entirely new classes, the model classifies by calculating the closest similarity measurement (Chen et al., 2019a Naik and Mammone; 1992). Formally, in few-shot learning, the training set includes many classes, each with multiple samples (Li et al.] 2018 |enn et al.) 2018). For example, C' classes are randomly selected from the training set and K samples from each category (totaling C x K samples) are used as input the model. A batch of samples from the remaining data in these C' classes is then used as the model's 's prediction target (batch set). The model must learn distinguish these C' classes from C x K pieces of data, task known C-way K-shot problem.
Point Cloud Completion. This task involves an important objective of inferring the complete shape of an object or scene from incomplete raw point clouds. Recently, numerous deep learning approaches have been developed to address this problem. For example, PCN (Yuan et al., 2018) extracts global features directly from point clouds and generates points using the folding operations from FoldingNet (Yang et al.] 2018). iZhang et al. (2020b) proposed extracting multiscale features from different network layers to capture local structures and improve performance. Attention mechanisms, such as the Transformer (Vaswani et al. (017), excel in capturing long-term interactions. Consequently, SnowfaakeNet (Xiang et al.] (021). PointTr (Yu et al. 202Tb). and SeedFormer (Zhou et al. 2022) emphasize the decoder component by incorporating Transformer designs. PointAuN (Wang et al. 2022) is built entirely on Transformer foundations. Recently, [in et al. (2023b) introduced a HyperCD loss for training neural networks that defines traditional CD in hyperbolic space. Furthermore, Lin et al. 2023a) proposed an InfoCD loss by incorporating the contrastive concept into the CD formula.
Gumbel Distribution. In machine learning and computer vision, the Gumbel distribution has been widely used in sampling methods (Maddison et al. 2014 Kool et al. 2019) and reparameterization techniques (Hujjben et al.) (002) Kusner and Seemández-Lobateee 2016 Potapczynski et al. 2020). For example, Hancock and Khoshgoftaar (2020) introduced the Gumbel-Soffman reparameterization to enable differentiable sampling from a discrete distribution during backpropagation in neural networks.
3

---

Published as a conference paper at ICLR 2025
3 APPROACH
3.1 PRELIMINARIES
[x P,, { P2 two of points (or items) that are sampled from two unknown distributions P1, P2 respectively, and K as the number of nearest neighbors considered for each point. Also, we refer set, and as the f2-norm of a vector. Given these notations, our goal is to predict the set-to-set similarity, stX,, X2), based on the conditional probability p(Pi = ,
Notations & Problem Definition. We denote X;
as the cardinality of
Gumbel Distributions. Recall that the Gumbel distribution is used to model the distribution of the maximum (or minimum by replacing the maximum with the negative minimum) number of samples from various distributions. Here we list the definition of a Gumbel distribution as follows:
Definition (Gumbel Distribution). The probability density function (PDF) of Gumbel distribution with parameters js TR, o denoted as Gumbel (p., σ), for random variable E R. is defined as
p(x) = expl +
where 2 x-
3.2 GPS: GUMBEL PRIOR SIMILARITY FOR SET-TO-SET MATCHING
Distributional Signatures. Given two sets of points X, Xg, we define the of Euclidean distances from each point in one set to its KNNs in the other set as their distributional signature:
P( = ( (x1,)) = || - Z2 |l. dnd (x2,g) = || x1.j
Ak E [K], Vi,.
(2)
where is (resp. jk) denotes the index of the k-th nearest neighbor X₂ (resp. X1) for 11,1 (resp. 222)), leading to unordered set of K(| + (x2) values.
Probabilistic Modeling. To compute GPS, we introduce the Gumbel distributions and distributional signatures as latent variables, shown in Figure 3 and propose probabilistic framework follows:
p(P P2 x2)
Σ
Σ
p(P = PP.. 4, dmmi X2)
qEQ dman
Σ
Σ
p(g)p(PI P2 g)p(dmin X1,
(3)
Q dman ED
where q Q denotes Gumbel distribution. Fig-
ure Biluussustsstsss the probability decomposition where Gumbel distribution q is selected for measuring set-to-set similarity based on the distributional sig- natures between sets. Note that the distributions of each k-th nearest neighbor distances could be modeled using mixture of M independent Gumbel
Pr Pz
q
dmmi
X, Xz
M
K
Figure 3: Graphical model for computing the conditional probability p(P: = Pa X', x2)
distributions. Consequently. Equation (3) further rewritten follows:
can be
p(P1 = P2 X2) x
Σ p (di (21,)); an 3k,m)
min
Ep
(1))) min (I2 ark Bkom
where & E [K], E [M] and {ax,m, 3m.m denote the predefined Gumbel parameters of the m-th mixture for fitting the k-th nearest neighbor distances. For simplicity. we use the same mixture for both sets (but they be different). Note that here take p(q) and p(P, Pa g) as constants with no prior knowledge.
reparametrization with Minimum Euclidean Distances. The KNN distances sets of (condi- tional) minima, not maxima. Therefore, we need to convert these minimum distances to some form of maximum values that can be modeled using Gumbel. Considering both computational complexity and
4

---

Published as a conference paper at ICLR 2025
14000
0.4
4000
0.4
12000
10000
0.3
Counts
6000
Probabiltt
Probabilty
2000
6000
0.1
1000
2000
0
10
-2
-log(min_dee 1ds++
Aog/min die)
Figure 4: Illustration of the distributions of minimum distances (with the learned best model) at test time for (Ieft) point cloud completion and (right) few-shot image classification.
learning objectives, we propose reparametrizing x def log (dmin + 6) in Equation where d > denotes a distance shift (see Section 3.3 for more discussions), and rewrite Equation accordingly as follows, which formally defines our GPS:
def
Σ (k.m) min
(I1,s)
(M) mén
(I1 + Σ
D (k.m) min
(x2,)) expl - - (m25)} (5)
(k.mm) min
where Pkkm
log ammm
Ba.m
kk.m
Ba
O, ak = "k
> 0, Bk.m op.m
min (x2,5) + Ōe.m
0, D (k.m)
(2115)
a..m (k.m))
(I1,)
8k,m
Bx.m
(). D min
(x2.5) a4km amnn
(ke.m)
9p.m
respectively.
3.3 ANALYSIS
Proposition 1. Letting f(x) =(= 0, then f has a unique maximum at x 1, and is concave when 0 < x < 2, otherwise convex when x 2,
Proof. By taking the first-order derivative of f. Vf(x) = (1 z)e-= and setting O, we can show that I has a unique extremum at x 1. By taking the second-order derivative, f² -(2- =, we can easily show its convexity and concavity by looking at the signs the values.
Gradients decrease (exponentially) when minimum distances approach the mode. This can be easily found from the form of the gradient in the proof above. As result, the contribution of the gradient from a data sample backpropagation will be smaller if its minimum distance closer to the mode, making it less important in training. In other words, the training will focus more on correcting poor predictions by adaptively assigning higher weights backpropagation. Note that all a's and 3 in GPS have an impact on the gradients in learning.
Distance Shifting. many applications, such as point cloud completion, the objective minimize the distances between predictions and the ground truth. In these scenarios, the mode of Gumbel
>> 0. To maximize similarity during training, all minimum distances should be enforced approach this mode. Therefore, our reparametrization includes d,uin + 6R because, as dmin approaches 0, log(dmin) would tend toward infinity, contradicting the goal of maximizing similarity by approaching the mode. However, - logg (dmin Ōx.m) avoids this problem. Figure (left)) shows example using randomly sampled airplane instance from ShapeNet (see our experiments on point cloud completion for more details), where all minimum distances are approaching 0 as well the Gumbel mode, achieving both goals of minimizing distance and maximizing similarity simultaneously.
distribution is expected to be O, which results in dz = (ak.m)
Similarity Embedding, Also in many applications, such as few-shot image classification, the objective is to minimize some loss, e.g., cross-entropy for classification, that depends on set-to-set similarity. In such cases, the minimum distances are not necessarily small possible, neither matching with the Gumbel distributions, as long as they help distinguish the data towards minimizing the loss, e.g., higher similarities for positive pairs and lower for negative pairs. Therefore, do
5

---

Published as a conference paper at ICLR 2025
Table 1: and 3 search with 1st nearest neighbors for few-shot classification (%) on CIFAR-FS.
0.5
1
2
2
2
2
2
2.5
3
0.5
0.5
1
1.5
0.5
3.5
2
2
Ace
72.75
72.51
73.23
73.54
73.67
73.14
73.08
74.22
72.14
73.23
73,89
73.71
73.52
74.14
74.17
73,80
Table a and [3 search using 2nd nearest neighbors performance (%) on CIFAR-FS.
0.1
0.5
2
2
2
2.5
3,5
0.5
0.5
2.5
3.5
4
Acc
71.67
72.84
73.17
73.04
73.08
73.66
72.45
72.85
73.23
72.67
73.37
72.95
73.44
60.74
not need distance shifting and simply set 8 = O. Figure 4 (right) illustrates a distribution of learned minimum distances for few-shot image classification (see our experiments for more details), compared with the Gumbel distribution used in training, where all the minimum distances are close to 1.
Table 3: Performance (%) using mixtures of Gum- bels and NNs on CIFAR-FS.
4 EXPERIMENTS
4.1 FEW-SHOT IMAGE CLASSIFICATION
1st NN
(5. 2)
(3.2)
(3.2)
(2.5.2)
(3,2)
(a,8)
(2,2)
(2,2)
(3.2)
(5.2)
We conduct our experiments on five benchmark datasets: minilmageNeet (Vinyals et al. 2016), tieredlmageNet (Ren et al. 2018). Fewshot- CIFARI00 (Oreshkin et al. 2018), CIFAR-
2nd NN
(3.2)
(2,5,2)
(4.
(2,
(4,2)
(a, B)
(4.2)
(2.2)
(4.
(1,
Acc
73.89
74.17
74.09
74.12
73.62
FewShot (Bertinetto et al. 201 8), and Caltech-UCSD Birds-200-2011 (Wah et al.
2011 We
follow the standard protocols in the literature split these datasets for our training and testing.
Network Framework. We take the framework in DeepEMD (Zhang et al. 2020a for fair compar-
isons with different losses. Following the literature (Sun et al. 2019: Chen et al. 2021 Liu et al.. 2021 Chikontwe et al. (022). we employ ResNet-12 12 as our network backbone. We notice that DcDDEMD and its extensions have different implementations. Precisely, we use the DeepEMD-FCN network for comparisons and report our results in our experiments. We also observe that GPS with the other implementations DeepEMD can still significantly improve the performance, For instance, by replacing FCN with Grid and Sampling layers (Zhang al. (022). our GPS in 1-shot 5-way can improve 1.2% over DeepEMD with Grid and 1.3% with Sampling.
Training Objective. Same as DeepEMD, we optimize the following objective for the few-shot tasks:
min Σ (Ki(Kus Xo), Mur Noiw), s.t. Xr (I (),, ()).));))}
where Ta,i; Pu.j stand for two patches from the u-th and v-th images with labels yus ye, respectively, for neural network parametrized by 6, and E for loss parametrized by w. In GPS, k uses KNNs to determine the matches between patches, while in DeepEMD a differentiable EMD was proposed find the matches. We reuse the framework by replacing the EMD with GPS.
Following DeepEMD, we re-implement all pre-training, meta- meta-training, valida- tion and testing stages with different loss functions, and retrain the networks from scratch. We keep all the hyperparameters the same as DeepEMD but fine-tune the loss-related hyperparameters, if exist, to report best performance. For better comparison, rerun the public code and report our results whenever possible; otherwise, we cite the original results for the remaining methods. In each result table, the tor-performing result is highlighted in bold. We conduct all experiments on server with 10 NVIDIA RTX 6000 11G GPUs and another one with 10 NVIDIA Quadro RTX 6000 24G GPUs.
Training Protocols.
Ablation Study. We have conducted some experiments for few-shot image classification, including:
M Gumbels, K Nearest Neighbors, and Mixtures: Table Table and Table list our hyperparam- eter search results for 1st NN only, 2nd NN only, and mixtures of Gumbels and NNs, respectively. With proper tuning, the best results under different settings are close to each other, and the mixtures seem to more robust to the data, leading similar performance with different combinations.
https://gi thw.. http://.ccco//cco///e/eeeee.

---

Published as a conference paper at ICLR 2025
6SWD
0.415
Ours
ASWD
0.26s
PWD
0.23s
Accuracy
EMD (OpenCV)
SWD
0.18s
EMDXOpenCV)
0.17s
InfoCD
0.16s
HyperCD
Our
0.06s
CD
0.04s
Epochs
0.50
0,05
0.10
0.5
0.20
0.25
0.30
0.35
0.40
0.45
(a)
Figure 5: Comparison on (a) validation accuracy during training, and (b) running time per iteration.
Table 4: Cross-domain results (%) (mindmagenet CUB) on I-shot 5- and 5-shot 5-way tasks.
Model
I-shot
5-shot
ProtoNet
50,01
72.02-10.67
MatchingNet
51.65+0.84
69.14
cosine classifier
44.17:1078
69.0110.74
linear classifier
50.37+0.79
73.3010.69
KNN
50.8441..1
71.25:10.99
DeepEMD
51.07:10.59
75.27
51.32:10:46
75.61
HyperCD InfoCD
51.76:10.48
75.48:
51.95++0.65
76,23++0.39
Figure 6: Visual matching results by (top) our GPS and (bottom) DeepEMI..
Ours: GPS
52.38:10.47
76.59
Training Behavior & Running Time: Figure (a) shows the validation accuracy vs. the number of epochs, and our approach consistently outperforms the other two competitors. Figure 5 (b) shows the running time with different loss functions to train the network, where our GPS (using Gumbel and 1st NN only) the only one that can manage to preserve similar computational time CD. Visualization of Matching Results: To understand the matching behavior, we visualize some matching results in Figure Compared with DeepEMD, our approach can effectively establish correspondence between local regions more precisely with just simple nearest neighbor search.
Comparisons with Different Losses. Table Table 5 and Table summarize our comparison results on few-shot image classification on different datasets with = 5.3 2 and NN only. As we see, our approach outperforms all the competitors on all the datasets, and is more robust than the others, in general, which demonstrates the effectiveness and efficiency of GPS.
4.2 POINT CLOUD COMPLETION
Datasets & Backhone Networks. We conduct our experiments on the five benchmark datasets: PCN
Yuaan et 2018 Shapehet-555/33 (Yu et al. (0211)) ShapeNet-Part (Yi ct al. 2016). and KITTI Geiger et al. 2012). We compare our method using seven different existing backbone networks, i.e.
FoldingNet Yang et al. 2018 , PeP-Net (Wen et al. 2021), PoinTrr Yu al. 2021b). SnowflakeNet (Xiang et al. 20 Ce-Net (Lin et al., (022). PointAut (Wang et al. [022] and SeedFormer (Zhou
2022). by replacing the CD loss with our GPS wherever it occurs. ct al.
Training Objective. Similar to InfoCD (Lin et al. 2023a). we optimize the following objective:
max Σ ER O))
where (K, denotes a pair of a partial point cloud and its ground-truth complete point cloud and is a network (e.g., one of the seven backbone networks) parametrized by 0.
Training Protocols. We use the same training hyperparameters such as learning rates, batch sizes and balance factors the original losses for fair comparisons. We conduct experiments
7

---

Published as a conference paper at ICLR 2025
Table 5: Results of 5-way (%) on minilmageNet and tieredImageNet datasets.
Method
mindmageNet
lieredlmageNet
I-shot
5-shot
I-shot
5-shot
cosine classifier (Chen al.
2019b
55.43.10.31
71.18:10.1
61.49
82.37-10.67
ProtoNet (neell et al. 2017
60.37.10.83
78.02-10.57
65.65
83,40:10.65
Matching Vinyals et al.
2016
63.08
75.99=0060
68.50-0.92
80.60+0.71
DeepEMD Zhang al.
2020a
63.36.10.75
79.15:20.66
0..48
83.89-0.67
CD
63.40:10466
79.54-10.9
70.23:0.64
84.01=0.31
PWD
Rowland et al.
2019)
63.92:1.777
78.77-0.37
70.6910.92
83,8810.34
SWD
Kolour et al. 2019
63.1510..66
78.46110.4
69.72:0.93
83.02-0.33
GSWD
Koloun ct al. 2019
63.66+0072
78.92+0047
70.25+0.86
83.62+0.31
ASWD
Nguyen ct al. 2021
63.16+0.75
78.87+0.45
69.30+0.91
83.71+0.38
HyperCD (Lin ct al. 2023b
63.63+00.55
79.78+0073
70.58+0.81
84.27+0.48
InfoCD (Ln et al. 2023a
64.01=0.32
80.87+0064
70.97+0.59
84.54+0.36
Ours: GPS
66.27=00.77
81.19+0.47
73.16+0043
85.52:
Table 6: 1-shot 5-way and 5-shot 5-way performance comparison (%).
CIFAR-FS
Caltech-UCSS) Birds
Method
5-shot 1-shot
1-shot
5-shot
1-shot
5-shot
ProtoNet
41.54=0.76
50.06
66.09
02.5
DeepEMD
71.16:10.42
85.1210.29
43.93-10.82
61.01:1.41
67.0410.31
85.12:10.76
CD
71.75
85,4810.51
44.15=10.66
61.22:10.58
67.1110.46
85.3110.59
PWD
71.58+0.31
84.76+ 0.33
43.47=-0.29
60.41+0.41
65.60+-0.33
84.87++0.45
SWD
70.99+00.34
84.52+00.7
43.32++0.28
60.31+0.31
65.44+0.32
84.38+0041
ASWD
71.45=+0.32
85.26+00.3
43.83=+0229
60.89+00.4
65.45++0.34
84.76++0.39
HyperCD
72.02.10.41
85.7710.43
44.42.10.33
61.5910.74
67.4511.38
85.42-10.75
InfoCD
72.31110.2
85,9110.339
44.81
61.8710.66
67.82.10.42
85,90100.68
Ours: GPS
74.22:10.22
86.9811.22
46.75
12.91
69,4210.33
86.7810.62
server with 4 NVIDIA A100 80G GPUs and one with 10 NVIDIA Quadro RTX 6000 24G GPUs due the large model sizes of some baseline networks.
Table 9: Performance com-
parison on ShapeNeet
Evaluation. Following the literature, we evaluate the best performance of all the methods using vanilla CD (lower is better). We also F1-
Losses
L2-CDx10'
Score @ 1% (Tatarchenko et al.) 2019) (higher is better) to evaluate the performance on ShapeNett55/34, For KITTI, we utilize the metries of Fidelity and Maximum Mean Discrepancy (MMD) for each method (lower better for both metrics).
LI-CD
4.16.10.028
L2-CD
4.82.10.117
DCD
5.7410.049
PWD
14.39=0.024
Grid Search on (a, 3). Table summarizes our search results. As we discussed in Section 3.3] all the settings produce similar performance. Considering the running time and hyperparameter search results in few-shot classification, in our experiments on point cloud completion, we only use 1st NNs and single Gumbel distribution to fit the data, without further tuning.
SWD
6.28=0.073
GSWD
6,26=0.034
ASWD
7.52+00.58
HyperCD
4.03+00.077
InfoCD
4.01
GPS
0.04
State-of-the--rt Result Comparisons. We summarize our comparisons as follows:
ShapeNet- Part: We begin by presenting our performance comparison in Table the ShapeNet- Part dataset using CP-Net. Evidently, our approach outperforms all other competitors.
NetpeNet-34: We evaluate performance across 34 seen categories (used during training) and
21 unseen categories (not used during training), detailing our results in Table 8 It is evident that our approach consistently improves the performance of baseline models, indicating its high generalizability for point cloud completion tasks.
ShapeNer-55: We evaluate the adaptability of our method across datasets for tasks with greater diversity. Table II lists the L2-CD across three difficulty levels, along with the average. Following the literature, we present results for five categories (Table, Chair, Plane, Car, and Sofa) with over 2,500 training samples each, shown in the table. Additionally, we provide results using the
8

---

Published as a conference paper at ICLR 2025
Table a and / search with 1st nearest neighbors on ShapeNet- using CP-Net.
0x
1
0.5
2
4
2
18
0.4
0.8
1.2
1.6
2
1
1
1
2
CD
4.29
3.98
3,94
4.04
4.34
4.68
4.11
4.05
4.13
4.72
Table 8: Results on ShapeNet-34 using L2-CD x 1000 and Fl score.
seen categories
21 unseen categories
Methods
CD-S
CD-M
CD-H
Avg.
F1
CD-S
CD-M
CD-H
Avg.
F1
NoldingNet
1.86
1.81
3.38
2.35
0.139
2.76
2.74
5.36
3.62
0.095
HyperCD FoldingNet
1.71
1.69
3.23
2.21
0.148
2.55
2.59
5.19
3.44
0.122
InfoCD FoldingNet
1.54
1.60
3.10
2.08
0.177
2.42
2.49
5.01
3.31
0.157
Ours: GPS FoldingNet
1.49
1.55
3.02
2.02
0.182
2.39
2.43
5.00
3.27
0.159
PoinTr HyperCD + PoinTr
0.76
1.05
1.88
1.23
0.421
1.04
1.67
3.44
2.05
0.384
0.72
1.01
1.85
1.19
0.428
1.01
1.63
3.40
2.01
0.389
InfoCD + PoinTr
0.47
0.69
1.35
0.84
0.529
0.61
1.06
2.55
1.41
0.493
Ours: GPS + PoinTr
0.43
0.64
1.27
0.78
0.533
0.60
1.04
2.52
1.38
0.495
SeedFormer
0.48
0.70
1.30
0.83
0.452
0.61
1.08
2.37
1.35
0.402
HyperCD SeedFormer
0.46
0.67
1.24
0.79
0.459
0.58
1.03
2.24
1.31
0.428
InfoCD + SeedFormer
0.43
0.63
1,21
0.75
0.581
0.54
1.01
2.18
1.24
0.449
Ours: GPS+ SeedFormer
0.42
0.61
1.20
0.74
0.582
0.52
1.00
2.15
1.22
0.451
F1 metric. Once again, our approach significantly improves the baseline models, especially with
simpler networks. We also include some visualization results in Figure
KITTI: To validate the effectiveness of GPS for point cloud completion on large-scale real- world benchmark, we follow (Xie et al., 2020) to finetune two baseline models with GPS on ShapeNetCars (Yuan al. 2018) and evaluate their performance on KITTI. We report the Fidelity and MMD metrics in Table 10 observing that GPS consistently improves the baseline models.
PCN: We also summarize the results on another benchmark dataset, PCN, with additional backbones in Table 12 showcasing state-of-the-ard performance in point cloud completion.
Table 10: Results on KITTI in terms of the fidelity and MMD metrics.
FoldingNet
HyperCD+F.
InfoCD+R
GPS+F.
PoinTr
HyperCD+P.
InfoCD+P
GPS+P.
Fidelity 4
7.467 0.537
2.214 0.386
1.944 0.333
1.883
0.000
0,000
0,000
0.000
MMD1 1
0.302
0.526
0.507
0.502
0.449
CONCLUSION
In this paper, present GPS, novel, effective, and efficient similarity learning framework for set-to-set matching. Our method fits predefined Gumbel distribution to the negative log minimum distances between set items. Originating from probabilistic graphical model, approach allows multiple Gumbel distributions to model the distributions of KNN distances as distributional signatures. We demonstrate superior performance in few-shot image classification and point cloud completion compared to traditional distance metrics, while maintaining similar running time to CD.
Limitations. Finding optimal hyperparameters for Gumbel distributions fit the distance distri- butions can be time-consuming, particularly when dealing with large sets, such as 16,384 points per object PCN and ShapeNet-55/34. In future work, we plan develop general guidelines for efficiently determining these hyperparameters.
ACKNOWLEDGMENT
Vijaya B. Kolachalama supported by grants from the National Institute on Aging's Artificial Intelligence and Technology Collaboratories (P30-AG073104 104 & P30-AG073105), the American Heart Association (20SFRN35460031) and the National Institutes of Health (R01-HL159620, RO1- AG062109, & RO1-AG083735), Venkatesh Saligrama supported by the Army Research Office Grant W911NF211024 AFRLGrant 1A8650-22-C1033 the National Science Foundation grants CCF-2007350, CCF-1955981 and CPS-2317079.

---

Published as a conference paper at ICLR 2025
Table 1I: Results on ShapeNet-55 using L2-CDx1000 and Fl score.
Methods
Table
Chair
Plane
Car
Sofa
CD-S
CD-M
CD-H
Avg.
F1
Net
2.53
2.81
1.43
1.98
2.48
2.67
2.66
4.05
3.12
0.082
HyperCD> + Net
2.35
2.62
1.25
1.76
2.31
2.43
2.45
3.88
2.92
0.109
InfoCD FoldingNet
2.14
2.37
1.03
1.55
2.04
2.17
2.50
2.71
0.137
Ours: GPS FoldingNet
2.07
2.30
1.02
1.47
2.01
2.13
2.44
2.64
0.143
PoinTr HyperCD PoinTr
0.81
0.95
0.44
0.91
0.79
0.58
0.88
1.79
1.09
0,464
0.79
0.92
0.41
0.90
0.76
0,54
0.85
1.73
1.04
0,499
InfoCD + PoinTr
0.69
0.83
0.33
0.80
0.67
0.47
0,73
1,50
0.90
0.524
Ours: GPS + PoinTr
0.61
0.79
0.31
0.76
0.64
0,41
0.68
1.44
0.84
0.529
SeedFormer
0.72
0.81
0.40
0.71
0.50
0.77
0,92
0.472
HyperCD SeedFormer
0.66
0.35
0.83
0.64
0.47
0.72
0.86
0.482
InfoCD SeedFormer
0.65
0.72
0.62
0.43
0.71
1.38
0.84
0.490
Ours: GPS SeedFormer
0.63
0.30
0.61
0.42
0.69
0.82
0.493
Table 12: Per-point LI-CD x1000 (lower is better) on PCN.
Methods
Plane
Cahinet
Car
Chair
Lamp
Couch
Table
Boat
Avg.
FoldingNet Yang al. 2018
9.49
15.80
12.61
15.55
16.41
15.97
13.65
14.99
14.31
HyperCD FoldingNe
7.89
12.90
10.67
14.55
13.87
14.09
11,86
10.89
12.09
InfoCD + FoldingNet
7.90
12.68
10.83
14.04
14.05
14,56
11.61
11.45
12.14
Ours: GPS + FoldingNet
7.38
12.61
10.46
13.12
11.92
13.39
10.86
10.59
11.30
PMP-Net Wen al. 2021
5.65
11.24
9.64
9.51
6.95
10.83
8.72
7.25
8.73
HyperCD PMP--Ne
5.06
10.67
6.83
11.01
8.18
7.03
8,40
InfoCD + PMP-Net
4.67
10.09
8.87
8.59
6.38
10.48
7.51
6.75
7.92
Ours: GPS + PMP-Net
4.52
10.02
8.80
8.45
6.31
10.42
7.46
6.70
7.84
PoinTr Yu al. a 2021b
4.75
10.47
8.68
9.39
7.75
10.93
7.78
7.29
8.38
HyperCDD Poinr
4.42
9.77
8.22
8.22
6.62
9.62
6.97
6.67
7.56
InfoCD Poin'I
4.06
9.42
8.11
7.81
6.21
9.38
6.57
6.40
7.24
Ours: GPS PoinTr
4.03
9,39
8.03
7.78
6.18
9.33
6.56
6.38
7.21
SnowtlakeNet (Xiung al. 2021
4.29
9.16
8.08
7.89
6.07
9.23
6.55
6.40
7.21
HyperCD SnoowwakkeNet
3.95
9.01
7.88
7.37
5.75
8.94
6.19
6.17
6.91
InfoCD SnowflakeNet
4.01
8.81
7.62
7.51
5.80
8.91
6.21
6.05
6.86
Ours: GPS SnowflakeNet
3.97
8.79
7.61
7.47
5.73
8.85
6.15
6.03
6.82
PointAuN Wang et al. (022
3.87
9.00
7.63
7.43
5.90
8,68
6.32
6.09
6.86
HyperCD + PomnAANN
3.76
8.93
7.49
7.06
5.61
8.48
6.25
5.92
6.68
InfoCD PointAuN
3.72
8.87
7.46
7.02
5.60
8.45
6.23
5.92
6.65
Ours: GPS PointAttN
3.70
8.83
7.42
7.00
5.59
8.43
6.22
5,91
6.63
SeedFormer (Zhou et al. 2022
3.85
9.05
8.06
7.06
5.21
8.85
6.05
5.85
6.74
HyperCD SeedFormer
3.72
8.71
7.79
6.83
5.11
8.61
5.82
5,76
6.54
InfoCD SeedFormer
3.69
8.72
7.68
6.84
5.08
8.61
5.83
5.75
6.52
Ours: GPS SeedFormer
3.68
8.69
7.65
6.80
5.05
8.55
5.72
5.63
6.48
2
Figure 7: Row-1: Inputs incomplete point clouds. Row-2: Outputs of Seedformer with CD. Row-3: Outputs of Seedformer with GPS. Row-4: Ground truth.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jen-Mei Chang, Michael Kirby, and Chris Peterson. Set-to-set face recognition under variations in pose and illumination. In 2007 Biometrics Symposium, pages 1-6. IEEE, 2007.
Sanping Zhou, Jinjun Wang, Rui Shi, Qiqi Hou, Yihong Gong, and Nanning Zheng. Large margin learning in set-to-set similarity comparison for person reidentification. IEEE Transactions on Multimedia, 20(3):593-604, 2017a.
Yuki Saito, Takuma Nakamura, Hirotaka Hachiya, and Kenji Fukumizu. Exchangeable deep neural networks for set-to-set matching and learning. In European Conference on Computer Vision, pages 626-646. Springer, 2020.
Mateusz Jurewicz and Leon Derezynski. Sel-to-sequence methods in machine learning: a review. Journal of Artificial Intelligence Research, 71
2021.
Tan Yu, Jingjing Meng, Ming Yang, and Junsong Yuan. object representation learning: A set-to-set matching perspective. IEEE Transactions Image Processing, 50:2168-2173 2021a.
Masanari Kimura, Takuma Nakamura, and Yuki Saito. Shift1 Fashion-specific dataset for set- to-set matching with several distribution shifts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3507-3512, 2023.
Ze Yang, Yinghao Xu, Han Xue, Zheng Zhang, Raquel Urtasun, Liwei Wang, Stephen Lin, and Han Hu. Dense reppoints: Representing visual objects with dense point sets, In Computer Vision-EEe ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28 2020, Proceedings, Part XXI 16, pages 227-244. Springer, 2020.
Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Few-shot image classifica- tion with differentiable earth mover's s distance and structured classifiers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12203-12213, 2020a.
Siyuan Yang, Jun Liu, Shijian Lu, Er Meng Hwa, and Alex C Kot. One-shot action recognition via multi-scale spatral-temporal skeleton matching. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Hao Zhu and Piotr Koniusz. Ease: Unsupervised discriminant subspace learning for transductive few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9078-9088, 2022.
Ali Saad-Eldin, Benjamin D Pedigo, Carey E Priebe, and Joshua Vogelstein. Graph matching via optimal transport. arXiv preprint arXiv:2111.05366, 2021.
Fangzhou Lin, Yun Yue, Ziming Zhang, Songlin Hou, Kazunori Yamada, Vijaya Kolachalama, and Venkatesh Saligrama. Infoed: A contrastive chamfer distance loss for point cloud completion. Advances in Neural Information Processing Systems, 36, 2023a.
Trung Nguyen, Quang-Hieu Pham, Tam Le, Tung Pham, Nhat Ho, and Binh-Son Hua. Point- distances for learning representations of 3d point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10478-10483 2021.
Mark Rowland, Jiri Hron, Yunhao Tang, Krzysztof Choromanski, Tamas Sarlos, and Adrian Weller. Orthogonal estimation of wasserstein distances. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 186-195. PMLR, 2019.
Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo Rohde. Generalized sliced wasserstein distances. Advances in neural information processing systems, 32, 2019.
Tong Wu, Liang Pan, Junzhe Zhang, Tai Wang, Ziwei Liu, and Dahua Lin. Density aware chamfer distance as a comprehensive metric for point cloud completion. In Advances in Neural Information Processing Systems, volume 34, pages 39088-29100. 29100, 2021.
Fangzhou Lin, Yun Yue, Songlin Hou, Xuechu Yu, Yajun Xu, D Kazunori Yamada, and Ziming Zhang. Hyperbolic chamfer distance for point cloud completion. In Proceedings of the IEEE/CVF international conference on computer vision, 2023b.

---

Published as a conference paper at ICLR 2025
Lillian Lee. Measures of distributional similarity. arXiv preprint cs/0001012, 2000.
Maximilian Ilse, Jakub Tomezak, and Max Welling. Attention-based deep multiple instance learning. In International conference on machine learning, pages 2127-2136. PMLR, 2018.
Oded Maron and Tomás Lozano-Pérez. framework for multiple-instance learning. Advances in neural information processing systems, 10, 1997.
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned- Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pages 945-953, 2015.
Baoguang Shi, Song Bai, Zhichao Zhou, and Xiang Bai. Deeppano: Deep panoramic representation for 3-d shape recognition. IEEE Signal Processing Letters, 22(12):2339-234 2015.
Arman Afrasiyabi, Hugo Larochelle, Jean-François Lalonde, and Christian Gagné. Matching feature sets for few-shot image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9014-9024, 2022.
Jonghyun Choi, Jayant Krishnamurthy, Aniruddha Kembhavi, and Ali Farhadi. Structured set matching networks for one-shot part labeling. In Proceedings the the IEEE Conference Computer Vision and Pattern Recognition, pages 3627-3636, 2018.
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In NIPS, 2016.
Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set trans- former: framework for attention- based permutation-inet conference on machine learning, pages 3744 3753. PMLR, 2019.
neural networks. In International
Masanari Kimura. Generalization bounds for set-to-set matching with negative sampling. In International Conference on Neural Information Processing, pages 468-476. Springer, 2022.
Wei- Bing Ma and Bangalore $ Manjunath. Texture features and learning similarity. In Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 425-430. IEEE, 1996.
Maria-Florina Balcan and Avrim Blum. On a theory of learning with similarity functions. In Proceedings of the 23rd international conference on Machine learning, pages 73-80, 2006.
Jie Yu, Jaume Amores, Nicu Sebe, Petia Radeva, and Qi Tian. Distance learning for similarity estimation. IEEE transactions on pattern analysis and machine intelligence, 30(3):451-46,, 2008.
Masoud Faraki, Xiang Yu, Yi-Hsuan Tsai, Yumin Suh, and Manmohan Chandraker. Cross-domain similarity learning for face recognition in unseen domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15292-15301, 2021.
Qiong Cao, Yiming Ying, and Peng Li. Similarity metric learning for face recognition. In Proceedings of the IEEE international conference computer vision, pages 2408-2415, 2013.
Jachoon Oh, Sungnyun Kim, Namgyu Ho, Jin-Hwa Kim, Hwanjun Song, and Se- Se-Young Yun. Understanding cross-domain few-shot learning based on domain similarity and few-shot difficulty. Advances in Neural Information Processing Systems, 35:2622-2633 2022.
Jen-Chun Lin, Wen-Li Li Wei, and Hsin-Min Wang. Demv-matchmaker: Emotional temporal course representation and deep similarity matching for automatic music video generation. In 2016 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 2772-2776. IEEE, 2016.
Wentong Liao, Michael Ying Yang. Ni Zhan, and Bodo Rosenhahn. Triplet-based deep similarity learning for person re-identification. In Proceedings of the IEEE International Conference Computer Vision Workshops, pages 385-393, 2017
Weiyang Liu, Zhen Liu, James M Rehg, and Le Song. Neural similarity learning, Advances in Neural Information Processing Systems, 32, 2019.
12

---

Published as a conference paper at ICLR 2025
Xi Cheng, Li Zhang, and Yefeng Zheng. Deep similarity learning for multimodal medical images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging Visualization, 6(3): 248 252, 2018.
Guixiang Ma. Nesreen K Ahmed, Theodore L Willke, and Philip S Yu. Deep graph similarity learning: A survey. Data Mining and Knowledge Discovery, 35:688-725, 2021.
Sanping Zhou, Jinjun Wang, Jiayun Wang, Yihong Gong, and Nanning Zheng. Point to set similarity based deep feature learning for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 3741-3750, 20177 2017b.
Mahmut Kaya and Hasan Şakir Bilge. Deep metric learning: A survey. Symmetry, 11(9):1066, 2019.
Qi Qian, Lei Shang, Baigui Sun, Juhua Hu, Hao Li, and Rong Jin. Softtriple loss: Deep metric learning without triplet sampling. In ICCV, pages $450-6458, 2019.
Chen Huang, Chen Change Loy, and Xiaoou Tang. Local similarity- deep feature embedding. Advances in neural information processing systems, 29, 2016.
Ismail Elezi, Sebastiano Vascon, Alessandro Torcinovich, Marcello Pelillo, and Laura Leal-Taixé. The group loss for deep metric learning. Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VII 16, pages 277-294 Springer, 2020.
Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew Scott. Multi-similarity loss with general pair weighting for deep metric learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2019.
Wei- Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. closer look at few-shot classification. arXiv preprint arXiv:1904.044: 04232, 2019a.
Boris N. Oreshkin, Pau Rodríguez, and Alexandre Lacoste. TADAM: task dependent adaptive metric for improved few-shot learning. 2018.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017.
Devang Naik and RJ Mammone. Meta-neural networks that learn by learning. 1992.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few shot learning. In ICMI., 2018
Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for supervisd few-shot classification. In ICLR, 2018
Wentao Yuan, Tejas Khot, David Held, Christoph Mertz, and Martial Hebert. Pen: point completion network. 3DV, 2018.
Yaoqing Yang, Chen Feng. Yiru Shen, and Dong Tian. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 206-215, 2018.
Wenxiao Zhang, Qingan Yan, and Chunxia Xiao. Detail preserved point cloud completion via separated feature aggregation. In European Conference on Computer Vision, pages 512-528. Springer, 2020b.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, and Zhizhong Han. Snowflakenet: Point cloud completion by snowflake point deconvolution with skip- stip-transformer. In ICCV, 2021.
13

---

Published as a conference paper at ICLR 2025
Xumin Yu, Yongming Rao. Ziyi Wang, Zuyan Liu, Jiwen Lu, and Jie Zhou. Pointr: Diverse point cloud completion with geometry-awaid aware transformers. In ICCV, 2021b.
Haoran Zhou, Yun Cao, Wenqing Chu, Junwei Zhu, Tong Lu, Ying Tai, and Chengjie Wang. Seedformer: Patch seeds based point cloud completion with upsample transformer. arXiv preprint arXiv:2207.103/5. 2022.
Jun Wang, Ying Cui, Dongyan Guo, Junxia Li, Qingshan Liu, and Chunhua Shen. Pointattn: You only need attention for point cloud completion. arXiv preprint arXiv:2203.08453 2022.
Chris J Maddison, Daniel Tarlow, and Tom Minka. A sampling. Advances in neural information processing systems, 27, 2014.
Wouter Kool, Herke Van Hoof, and Max Welling. Stochastic beams and where to find them: The gumbel-top-k trick for sampling sequences without replacement. In International Conference Machine Learning, pages 3499-3508. PMLR, 2019.
Iris AM Huijben, Wouter Kool, Max B Paulus, and Ruud JG Van Sloun. A review of the gumbel-max trick and its extensions for discrete stochasticity in machine learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(2):1353- 1371, 2022.
Matt Kusner and José Miguel Hernndez- Lobato. Gans for sequences of discrete elements with the gumbel-softmax distribution. arXiv preprint 1r 2016.
Andres Potapczynski, Gabriel Loaiza-Gama and John P Cunningham. Invertible gaussian reparam- eterization: Revisiting the gumbel-softtaxx Advances in Neural Information Processing Systems, 33: 33:12311-12321 2020.
John T Hancock and Taghi M Khoshgoftaar. Survey categorical data for neural networks. Journal of big data, 7(1):28, 2020.
Luca Bertinctto, Joao Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differen- tiable closed-form solvers. arXiv preprint arXiv: /85.00660 2018.
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011.
Qianru Sun, Yaoyao Liu, Tat-Seng Chua, and Bernt Schiele. Meta-transfer learning for few-shot learning. In Proceedings of the IEEE/CVF conference computer vision and pattem recognition, pages 103-412,777 2019.
Zitian Chen, Subhransu Maji, and Erik Learned-Miller. Shot in the dark: Few-shot learning with no base-class labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1668-2677) 2021.
Ge Liu, Linglan Zhao, Wei Li, Dashan Guo, and Xiangzhong Fang. Class-wise metric scaling for improved few-shot classification. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 586-595, 2021.
Philip Chikontwe, Soopil Kim, and Sang Hyun Park. Cad: Co-adapting discriminative features for improved few-shot classification. In Proceedings of the IEEF/CV Conference on Computer Vision and Pattern Recognition, pages 14554-14563, 2022.
Chi Zhang, Yujun Cai, Guosheng Lin, and Chunhua Shen. Deepemd: Differentiable earth mover's distance for few-shot learning- IEEE Transactions on Pattern Analysis and Machine Intelligence, 45355:5632-5648, 2022.
Wei- Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Wang, and Jia-Bin Huang. A closer look at few-shot classification. In ICLR, 2019b.
Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. 2017.
Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. scalable active framework for region annotation in 3d shape collections, ACM Transactions Graphies (ToG), 35(6):1-2,, 2016.
14

---

Published as a conference paper at ICLR 2025
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE conference on computer vision and pattern recognition, pages 3364-3361. IEEE, 2012.
Xin Wen, Peng Xiang, Zhizhong Han, Yan-Pei Cao, Pengfei Wan, Wen Zheng, and Yu-Shen Liu. Pmp-net: Point cloud completion by learning multi-step point moving paths. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7443-7452, 2021.
Fangzhou Lin, Yajun Xu, Ziming Zhang, Chenyang Gao, and Kazunori D Yamada. Cosmos propagation network: Deep learning model for point cloud completion. Neurocomputing, 507: 221-234, 2022.
Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. What do single-view 3d reconstruction networks learn? In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 3405-3414, 2019.
Haozhe Xie, Hongxun Yao, Shangchen Zhou, Jiageng Mao, Shengping Zhang, and Wenxiu Sun. Grnet: Gridding residual network for dense point cloud completion. ECCV, 2020.
15


Reviews:
[{"review_id": "rT8oN38r8H", "reviewer": "Reviewer_auE8", "summary": "The paper presents a new similarity metric for set-to-set matching problems. The proposed similarity metric leverages Gumbel distribution as a prior to model k-NN across sets. This formulation makes the proposed similarity metric more robust to outliers compared to other similarity metrics like Chamfer Distance while being significantly more efficient to compute compared to robust metrics like Chamfer Distance. \n\nThe proposed similarity metric is applied to model objective functions for two problem settings: few-shot classification, point-cloud completion. The proposed metric outperforms other metrics across all evaluations and is similar to Chamfer Distance in runtime.", "questions": "- Have you considered applying the metric for matching and registration problems such as point cloud registration? The paper mentions that \"Set-to-set matching aims to identify correspondences between two sets of unordered items by minimizing a distance metric ... \" however it is not clear to me how the proposed metric can be used to yield such correspondences. It might be valuable to add some discussion on potential formulations for such applications.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Set-to-set matching is fundamental to many problem settings, the proposed formulation is novel, theoretically well-motivated and empirically outperforms other similarity metrics in chosen application domains. This is a significant contribution of interest to the wider community and can have impact beyond the two applications. \n- Extensive evaluation and transparent reporting of influence of hyperparamters on results.  \n- I highly appreciate code submission.", "weaknesses": "- As authors have already pointed out, searching for hyperparameters for the proposed formulation is non-trivial. This might limit wider adoption of this metric.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H7qyRaAfiM", "reviewer": "Reviewer_45Az", "summary": "The paper presents GPS, a Gumbel-prior-distribution mechanism to train models that can compare sets and compute correspondences. The idea is to use the Gumbel distribution to model extrema (minima or maxima) of the matching scores; this is based on Extreme Value Theory. GPS also proposes the use of an offset to compensate the modeled scores and improve the modeling of the scores. The paper presents experiments on image classification and point cloud completion where the proposed method consistently show improvements over the considered baselines.", "questions": "1. Why is the use of $\\delta$ necessary from the theoretical point of view?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Motivation and description of the problem is quite clear and it is quite important. I agree that we need to investigate better solution for set-to-set matching. In particular, I agree that the use of the statistical Extreme Value Theory into ML/AI can bring benefits to many problems.\n- The clarity of the paper in general is clear. The narrative is clear and I think the paper can be reproduce w/ some reasonable effort.\n- The set of experiments is adequate and tackles different applications showing the benefits and applicability of the approach.", "weaknesses": "While I agree about the use of EVT or Gumbel distribution to tackle problems that require modeling extrema (minimum or maximum) of matching scores, I disagree with the use of particularly using the Gumbel distribution only. Here are the specific concerns:\n\n1. According to EVT, the distribution that generally models the extrema (minima or maxima) is the Generalized Extreme Value (GEV) distribution; the Gumbel distribution is a special case of the GEV distribution. Thus, it is possible that for some problems the Gumbel distribution is not the appropriate one and thus it can affect performance. Note however that EVT in general can only be applied when the number of scores is large, otherwise, the theory and thus the distributions of EVT (including Gumbel) cannot be applied, strictly speaking. This is something that I think the paper is not exploring nor stating.\n\n2. While it is true that Gumbel can model minima or maxima, it is not true that it can model order statistics. In other words, I think the narrative of the paper is correct only when it states that it models minima or maxima when comparing entities of a set. However, the Gumbel distribution cannot model order statistics, i.e., statistics of the 2nd, 3rd, or Kth scores. Given this, I find concerning that the paper uses it to model the k-th nearest neighbors using a mixture of Gumbel distributions (lines 200 - 203). EVT only models the minima or maxima, but not the order statistics. I don't see a justification in the paper describing rigorously the use of Gumbel distribution in this case. Unless, the introduction of $\\delta$ as shown in line 232 compensates for that. But unfortunately the paper does not justifies this well.\n\n3. The paper is missing prior work exploring EVT to model minima/maxima in matching procedures. Thus, I think the narrative overstates that this is the first work using statistical information for set-to-set matching. See references shown below. \n\n4. Incomplete experiments. While I think the applications used in the experiments are diverse, I think the experiments would've been more informative and more convincing if they could show that other models (e.g., a transformer, or CLIP) also benefit from the proposed GPS.\n\nReferences:\n1. Scheirer, Walter J., et al. \"Meta-recognition: The theory and practice of recognition score analysis.\" IEEE transactions on pattern analysis and machine intelligence 33.8 (2011): 1689-1695.\n2. Fragoso, Victor, et al. \"EVSAC: accelerating hypotheses generation by modeling matching scores with extreme value theory.\" Proceedings of the IEEE international conference on computer vision. 2013.\n3. Fragoso, Victor, and Matthew Turk. \"SWIGS: A swift guided sampling method.\" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013.\n\n-------\nPost-discussion\n\nAfter discussion with the authors, clarity about the motivation of using Gumbel distribution improved. I still think the paper could've  included a more solid justification, but the experiments show the method works. I would just recommend the authors to update the narrative justifying the design choices a bit better and I will raise my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "E3tqinlPUH", "reviewer": "Reviewer_S2Z9", "summary": "It proposes a novel probabilistic method for set-to-set matching called GPS (Gumbel Prior Similarity), based on distributional similarity and Gumbel distributions. This method measures the similarity between the underlying distributions generating the sets. This research is very meaningful and contributes a lot.", "questions": "I have some suggestions. The proposed method can perform better in set-to-set matching, demonstrating good performance in few-shot image classification and 3D point cloud completion. I am particularly interested in whether it can also achieve good performance in image matching and point cloud matching, as they also involve the set-to-set matching problem. If it performs well in these areas, it would further highlight the impact of your method. If possible, please consider adding relevant experiments.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method presents an innovative probabilistic approach to set-to-set matching based on distributional similarity and Gumbel distributions. The idea is quite novel and significant, as it is helpful for the training of differentiable neural networks and offers considerable value to the community.\n\n2. The experiments look promising, especially regarding the superior performance in few-shot image classification and 3D point cloud completion.\n\n3. The paper is well-written and easy to follow.", "weaknesses": "I don't see any obvious drawbacks, but I am concerned about the efficiency issues mentioned in your limitations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cIRo8IWy9x", "reviewer": "Reviewer_5tK4", "summary": "The paper proposes to calculate the similarity between sets based on the Gumbel prior distribution.", "questions": "There is a lack of in-depth theoretical discussion on some key issues. For example, in practical applications, the assumed independent and identically distributed conditions are difficult to strictly meet. The paper only shows that the assumption seems feasible through experimental results, but does not theoretically analyze the error range brought by this approximation and the potential impact on the performance of the method. This theoretical incompleteness may affect the reliability and universality of the method.\n\nThe experiments are mainly focused on a few common image classification and point cloud completion datasets. Although these datasets are representative, they cannot fully cover all possible application scenarios and data distributions.\n\nAlthough the paper claims that the GPS method has achieved better results than some existing methods in the experiment, a careful observation of the experimental data shows that in most cases, the performance improvement is relatively small. For example, the accuracy improvement on some datasets may be only a few percentage points, which may not be significant in practical applications, especially considering the additional computational cost and model complexity that may be introduced.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The presentation is nice and clear.\nVery interesting set of experiments and they are extensive.", "weaknesses": "The GPS method proposed in the paper calculates the similarity between sets based on the Gumbel prior distribution, which is, to a certain extent, an improvement on existing methods. However, the degree of this innovation is relatively limited. For example, within the framework of set matching and similarity learning, many studies have attempted to introduce new methods and ideas from different perspectives. The GPS method, which is based on distance distribution fitting and the use of Gumbel distribution, has not fundamentally broken through the scope of existing research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aFKuIhc8cA", "reviewer": "Reviewer_YjK9", "summary": "This manuscript concentrates on the task of set-to-set matching. To this end, motivated by the observation that the distributions of minimum distances fro CD can be accurately modeled by Gumbel distribuctions, it proposes GPS based on Gumbel prior distributions. The proposed method has been validated on tasks including few-shot image classification and 3D point completion, and the results demonstrate its significance.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The idea of GPS is novel and interesting, expecially considering that metrics like Chamfer Distance have been widely adopted in many research areas and replacing Chamfer Distance with GPS maintains the same linear computational cost\n\n2. The writing of this manuscript is acceptable\n\n3. The proposed method has been validated on different tasks, and the experimental results demonstrate its significance", "weaknesses": "1. The organization of this paper can be further improved. For example, on the first page, Fig.2 is placed before Fig.1. Fig. 2 (c) is kind of confusing for me, even if I have read related context for several times. \n\n2. In Section.3, Fig.3 doesn't help understand the methodology, but even make me more confusing. \nThere should have one image that helps understand the design of GPS better. In the current version, any of  Fig.1, Fig.2, Fig.3  doesn't help too much. I think there also lacks a figure demonstrating the whole paradigm of applying GPS to a specific task.\n\n3. For the experimental part, I think the task of image matching (SuperGlue, CVPR 2020) or point cloud matching (Predator, CVPR 2021) should also be considered, as establishing correspondences is a fundamental problem in computer vision.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)