Difference-of-submodular Bregman Divergence
Authors: Masanari Kimura
Takahiro Kawashima
Tasuku Soma
Hideitsu Hino
Venue: ICLR 2025 Poster (2025)
Keywords: Bregman Divergence
Permutation-invariant neural networks
Metric learning
Submodular functions
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: The Bregman divergence, which is generated from a convex function, is commonly used as a pseudo-distance for comparing vectors or functions in continuous spaces. In contrast, defining an analog of the Bregman divergence for discrete spaces is nontrivial. Iyer & Bilmes (2012b) considered Bregman divergences on discrete domains using submodular functions as generating functions, the discrete analogs of convex functions. In this paper, we further generalize this framework to cases where the generating function is neither submodular nor supermodular, thus increasing the flexibility and representational capacity of the resulting divergence, which we term the difference-of-submodular Bregman divergence. Additionally, we introduce a learnable form of this divergence using permutation-invariant neural networks (NNs) and demonstrate through experiments that it effectively captures key structural properties in discrete data. As a result, the proposed method significantly improves the performance of existing methods on tasks such as clustering and set retrieval problems. This work addresses the challenge of defining meaningful divergences in discrete settings and provides a new tool for tasks requiring structure-preserving distance measures.

Published as a conference paper at ICLR 2025
DIRERENGEE-OF-SUBOOUUAA BREGMAN DIVER- GENCE
Masanari Kimura"
Takahio Kawashima" ZOZO Research ZOZO Next, Inc.
School Mathematics and Statistics The University of Melboumne Victoria, Australia
Chiba, Japan
m. w..kimurauun immelmmll mmelm lb. edu. au
takahiro, kww.shimalzzzaooo....o...
Tasuku Soma & Hideitsu Hino
Institute of Statistical Mathematics /RIKEN AIP Tokyo. Japan
{soma, hino } sm. www.c.c..........m..mmm......... ac
ABSTRACT
The Bregman divergence, which is generated from convex function, is monly used as a pseudo-distance for comparing vectors or functions in continu- spaces. In contrast, defining analog of the Bregman divergence for discrete spaces is nontrivial. [yer & Bilmes (2012b) considered Bregman divergences on discrete domains using submodular functions as generating functions, the discrete analogs of convex functions. In this paper, we further generalize this framework to cases where the generating function neither submodular nor supermodular, thus increasing the flexibility and representational capacity of the resulting divergence, Bregman divergence. Additionally, we introduce learnable form of this divergence using permutation-invariant neu- ral networks (NNs) and demonstrate through experiments that it effectively cap- tures key structural properties in discrete data. As result, the proposed method significantly improves the performance existing methods on tasks such as clus- tering and set retrieval problems. This work addresses the challenge of defining meaningful divergences in discrete settings and provides a new tool for tasks re- quiring structure-preserving distance measures.
which we term the
INTRODUCTION
A divergence is principal concept that determines the geometric structure of space of interest, which is formally defined as follows:
Definition 1.1. Let be a set. function D 2 - R is called a divergence on Q if D satisfies the following conditions: for all z, y E Ω.
D(a w) 2 0, and ((x,y) + ==
The Bregman divergence class of divergences that measure dissimilarity between two points based on strictly convex function. Given strictly convex and differentiable function f. the Breg- man divergence D, between two points and y in convex set C Ed defined
Dx fxx f(y) (V/()),--))
where Vf(y) is the gradient of f evaluated at y and (-, -) denotes the inner product. It generalizes the squared Euclidean distance and the Kullback-Leihh divergence (Kullback Leibler 1951). Typically, divergences are defined differential manifolds, such Euclidean space and family
"qqual contribution.

---

Published as a conference paper at ICLR 2025
of distributions. Compared to these continuous spaces, it is more challenging to define reasonable and capable metric on discrete spaces, such as 2 with V = (1...., N being a finite ground set. Although there numerous choices for measuring distance similarity between subsets X, e 2V, most known metrics in the literature are constructed by counting the sizes of X E Y, X \ YY and other simple set operations (Choi et al. 2010). Such classical metrics easily become less meaningful when the ground set V is large because rarely do same elements co-occur in two subsets of V.
The submodular-Bregung the theory of submodular set functions. It is based on the fact that subgradients and supergradients can be defined for any submodular function. Therefore, Bregman- like divergence can be defined through a submodular function f in the way analogous to the standard Bregman divergence O
divergence (Iyer & Bilmes 2012b) is the first to tackle this problem with
Although the submodular-Br Breggan divergence is quite natural and intuitive, it is not clear whether it satisfies the definition of divergences. This problem comes from identifiability of the divergence: D(x, y) = 0 - -
y in Definition 1.1 In the usual Bregman divergence I the strict convexity is required for f to guarantee the identifiability. Similar to the usual Bregman divergence, the sub- gradients or supergradients are insufficient to make it divergence; it may be necessary to assume the existence of strict subgradients or strict supergradients.
Another issue in the abbnouular-Beegaan divergence is that the choice of the submodular function is ad-hoc. Although [yer & Bilmes (2012b) introduces several concrete examples of the submodular function, the resulting divergences are again the forms with respect to simple set operations. So is unclear whether they actually overcome the difficulty of the classical metrics between sets. There- fore, a more flexible framework for handling the submodular- -Brlgan divergence is necessary, and it would be even better if we could extend the capability of Boooodular--regnnee divergence.
Our contribution. In this paper, we introduce a novel class of divergences on discrete spaces that is strictly more expressive than submodular Bregman divergences and propose a learning framework of the divergences through permutation invariinn NNs. First, we formally show that the submodular- Bregman divergence is indeed a divergence if / is strictly submodular. By extending this observa- tion, we further propose a new class of divergences that can be defined even f is neither sub- modular nor supermodular. Since the concrete way to compose the divergence depends difeerence-of-ssummduule decomposition (Narasimhan & Bilmes 2005 [yer & Bilmes| 2012b] Li & Du 2020), we call it cetprrencceoo-suummodiiee power of class of the underlying set function f gets larger. We then propose a learnable form of difference- of-submodular Bregman divergences based on submodular permutation-invariaat NNs. Numerical experiment shows that learnable differeeceeoo-ssubmdddll Bregman divergences can capture the crucial structure and significantly improves the performance of existing methods in downstream tasks.
the
Bregman divergences. We show that the expressive Bregman divergences defined by set function f gets richer if the
Related work. Bregman divergences play a central role in various areas of statistics and machine learning, including clustering, regression, and optimization. For example, the classical k-means al- gorithm is a special case of Bregman k-means clustering, where the squared Euclidean distance is replaced with a Bregman divergence. This generalization allows for clustering based on different divergence measures, providing more flexibility (Banerjee et al., 2005). The Bregman divergence is closely related to the exponential family of distributions and generalized linear models. The di- vergence can be derived from the convex conjugate of the log-partition function of the exponential family (Wainwright & Jordan 2008). In information geometry, it is used to define the geometry of statistical models and provides a framework for understanding various divergence measures (Amari) 2016). The Bregman divergence is also appeared in matrix factorization techniques. such as non- negative matrix factorization with Bregman divergence, which allows different divergence mea- sures to be used in the factorization process (Cichocki Amari 2010). In optimization, mirror descent algorithms and other first-order optimization methods. These methods benefit from the divergence's properties to achieve better convergence (Beck & Teboulle, 2003). In variational inference, the Bregman divergence measures the difference between the true posterior distribution and the variational approximation (Blei et al., 2017). Faust et al. (2023) elaborates the theory of difference- ffference-of-coomee man divergences on a discrete space, it is expected that the wealth of knowledge provided by these studies can be utilized.
used in
algorithms through the lens of the Bregman divergence. By considering Breg-
2

---

Published as a conference paper at ICLR 2025
Distance metric learning (DML Ye et al. 2018 Wang & Sun 2014)) is technique used to learn a distance metric that can effectively measure the similarity or dissimilarity between instances. This is crucial for improving the performance of various machine learning tasks such as classifi- cation Weinberger et al. 2005 Davis et al. 2007; Goldberger et al. 2005). clustering (Xing et al, Ye al. 2007). and information retrieval (Song al. 2016 [Schroft al. 2015 Wang et al. 2014). Min et al.] (2009) and Salakhutdinov & Hinton (2007) used NNs to learn representations that make metric learning easier. There is also research in distance metric learning that aims to learn Bregman divergences from data (Li et al.) 2023; Rezaei et al. [023] Lu et al., Siahkamari et al. 2020). However, to the best of our knowledge, there no research that learns Bregman divergences over discrete sets.
2002
Kulis al. 2005
2023
2
PRELIMINARIES
In this section, we introduce submodular and supermodular functions and their differentials, the concept of striet modularity, and permutation-invariant NNs necessary for the flexible realization of submodular and supermodular functions, as the building blocks of the divergence proposed in this paper.
2.1 SUBMODULAR FUNCTIONS AND SEMIDIFERENTIALS
Definition 2.1. A set function : 2 IR is said to be submodular if
f(X) + f(Y) > f(X 5
holds for every X,Y V. addition, f supermodular if -f is submodular, and f is modular is both submodular and supermodular.
modular function m 2 - known to be written by = =xx))=Eee m(i), and it identified as the vector (mi, ) = (in(,,...,,I (N)) E R-- (we define m0) == 0). Thus, the inner product of two modular functions m, m' 2 E is calculated as the inner product of the two vectors: (m, m') = Σ mi)m'(()) Σi= m.m!..
Submodular functions are often understood as discrete versions of convex functions, Indeed, subd- ifferentials can be defined for a submodular function / (Fujishige 2005). Interestingly, superdiffer- entials also defined for submodular function (Iyer & Bilmes
2012b):
Definition 2.2. Let f 2V + R submodular function. The set of modular functions
,, = {m E R VX C V, m(X) -m(Y) < - (x))-(())
called the subdifferential of Y C V, and by E (Y) is called subgradient of f at Y CVV Similarly, the set of modular functions
of (Y) {m E R-W V V. m(X) m(Y) 2 - f(Y))
called the superdifferential of f Y S V. and gy E of (Y) called a supergradient of f at Y YCV V.
Sub- and super-differentials are together referred to semidifferentials (Iyer al. 2013). In gen- eral, there are infinitely many choices to pick subgradient by E @y(YY (see the proof of Propo- sition A.4) and the extreme point can be obtained by greedy algorithm (Edmonds) 1970). On the other hand, Iyer& & Bilmes (201b)) proposes three types of supergradients gy, .9.yy,90 gy E o (Y) and they are called grow, shrink, and bar supergradients, respectively (Iyer et al. 2013). Table shows the form of these supergradients.
2.2 STRICT SUBMODULARITY
Recall that Bregman divergences are defined using strictly convex functions, We analogously need introduce strict submodularity for set functions.
Definition 2.3. A set function f 2V + IR is said to be strictly submodular if
f(X) f(Y)
((x
(2)
3

---

Published as a conference paper at ICLR 2025
Table Supergradients proposed in (yer Bilmes, 2012b). We denote S(IV) = S(() f(Y) for any Y - V and V.
jEY
i Y JGYY f0 f6
Oy (r) (r gr FGIVAG)
for every non-comparable X,Y V and f is strictly supermodular if -f is strictly submodular.
For example, the facility location function is important class of submodular functions. Let Øsy be non-negative values for E V ,,,...,N)..... and k 1...., ......K K. Then, the facility location function
is written as fFC X + Ex= maxiex lsk- More generally, the log-sum-ex, relaxation of the facility location function,
FFC, (X)
Σ 1=1
log
coals
(3)
VEX
strictly submodular for > O (see Appendix A.2 for the proof) and converges to frc as E - 0.
For strictly submodular function,
define strict semidifferentials:
Definition 2.4. Let 2 R be strictly submodular function. The set of modular functions
(Y) {m E ERR vX E 2VM m(X) - mmY) < f(x) f(Y)}
(4)
is called the strict subdifferential of f Y CVV and hy o,(Y) is called strict subgradient of at Y C V. Similarly
of o'(Y) (m RM XX 2 (YY)), m(X) m(Y) f(X) f(Y))
(5)
called the strict superdifferential f at Y S V. and ğy E ås is called a strict supergradient of f at Y V.V.
The non-emptiness of Ô shown in Appendix A.1 All of the three supergradients .r. Øy, and gy defined in Table satisfy the definition of the strict supergradients if f is strictly supermodular.
Proposition 2.5 (Strict supergradients). The modular functions ğy gy, and gy defined in Table are all the strict supergradients if f is strictly supermodular.
See Appendix A.1 for the proof.
2.3 PERMUTATION-IN NEURAL NETWORKS
Permutation invariance is the key concept in leveraging modern NN architectures to model set- the index set and Xy = (xi,..., XM] E R XI,.... XM RR., M < N) be the entire set of matrices with N or fewer columns. We consider an NN JNN XN + V, where y is the output space?] The permu- tation invariance requests SNN behave set function on 2V.
structured data (Zaheer et al. 2017). Let = N)
Definition 2.6. The function fNN is said to be permutation invariant it satisfies
fNN([x]..... xM)) = fNN fNN((Xex(()) X..,,xx (x)(
for any M E V, any inputs ".....;M: and any permutation RM {1, (1...., M) (I....M)
X and Y are said to he non-comparable neither C Y nor C Thhe elements of the input X1,..., XM not necessarily real vectors, but we consider real vectors for simplicity.

---

Published as a conference paper at ICLR 2025
The permutation invariance and permutation invariant NN are first introduced along with Deep Sets intiaaaiiionnniiiaant architectures are developed such as PointNet (Qi a.l 2017). Set Transformer (Lee et al.) 2019), Perceiver (Jaegle et al. 2021), and
Zaheer et al., 2017). Today, many effective
SerVAE Kim et al. 2021). See (Kimura et al. 2024 Xie Tong 2025) for comprehensive reviews
about permutation inniiaatttoo--niiaant invariant NNs.
we define the In particular, PointNet is convenient for modeling submodular functions. As in B PointNet architecture in the generalized form with log-sum-exp.
Definition 2.7. Let Φ = [di...., ox] IRP + RK y RK - R. 2 O, and X S V..
When
> 0, the architecture of e-PointNet frN,e is written by
(<lo8 Σ
fxxle((x)lexx=== =Y
(x((x))
log Σ
(6)
RE
iEX
For 0, is defined as
fex.o((x(lex))> = (mmax (x;),
max Φx xx(xx) Te
TEX
In e-PointNet, the log cog-sum-xpp or max operations realize permutation invariance. For 0, fPN.e reduces to the vanilla PointNet (i.e., fPN.o) with the + if is continuous. Since Ø and are arbitrary functions, we can design them using NNs freely. Given X1,..., XN, [rx,e((x))Exx) is fe ([x]]iex) Especially, if >> 0, $1...., &x are all
regarded as the set function (X) non-negative function, and y the summation function (namely, 7xx = =, v.). e-PointNet fPN,
reduces to the relaxed facility location function 3 which leads to strict submodularity.
3 THEORETICAL FOUNDATIONS
3.1 BRMODULLAR-RRGGMAA DIVERGENCES
The generalized Bregman divergence is the variant of the Bregman divergence ① in which the gradient is replaced by a subgradient. [yer & Bilmes (20126) introduced submodular-Bregnat divergences as a subclass of the generalized Bregman divergence. The earlier work, however, did not explicitly mention that in order for the identifiability of the divergence to always hold, it is necessary for the submodular functions involved be strict. Complementarilyy, we consider how the submodular-Bregmann divergences become proper, satisfying Definition I.I Hereafter, we denote 1x as the indicator vector of set X V.V.
Theorem 3.1. Let f 2V + R. be strictly submodular function. For every Y C V, there exist modular functions hy.gy E E that make
D,(X,Y) = f(X) f(Y)) -hy, 1x-11x11- 1y),
and
-(x,))=-((X) + f(Y) + (or, 1x -r)
(8)
satisfy the conditions of divergences shown in Definition T.I respectively.
Proof. For every C V, we can take a strict subgradient of f at Y for hy, i.e., by E ŌY)). This choice of the modular function by leads
Dx Y) = fXX) - f(Y) - hy + hY 2 ((X) - ((V) f(x) ((Y)
for all X C V and the equality holds if and only if X = Y. This Dy satisfies the conditions
in Definition I.1 We can also show that Df satisfies Definition 1.1 with strict supergradient gy E (Y) in the same manner.
In Iyer& & Bilmes 2012b), Dy and D are referred to as lower-bound submodular Beanodular-BBeemaan diver- gence and upper-bound submodular-Bregee divergence D, depends the (strictly) subgradient map 7l Y hy oY(Y) since the choice of (strict) subgradient in Dy is not unique. Although the dependency should be clarified as D omit it for simplicity (do the same for D().
divergence, respectively. The submodular submotular-Bregman
5

---

Published as a conference paper at ICLR 2025
3.2 DISCRETE BREGMAN DIVERGENCES
Intuitively, the expressive power of f may directly affect the expressive power of the submodular- Bregman divergences 77). (B) (in fact, this is true and will be proved in Subsection (33] If can extend the function class that can be taken as f. the flexibility of the divergence may increase. Surprisingly, the submodulla-Bregman divergences ( (8) are well-defined even f is general set function not necessarily submodular (nor supermodular). The strong DS offferenceeo--submuullll decomposition (Li & Du 2020) is the key idea.
Theorem 3.2 (Strong DS Decomposition (Li Du. 2020 Every set function 2 can be decomposed into the difference of two monotone increasing and strictly submodular functions f' and f², i.e., f-
Note that set function f! 2 IR is said be monotone increasing when (Y) > ('
for any X C Y C V. The (weak) DS decomposition is introduced in (Narasimhan & Bilmes, 2005) and proved in (Iyer & Bilmes 20T2a) rigorously and combinatorially. The proof of strong DS decomposition provided in (Li & Du 2020) is constructed by tweaking that in (Iyer & Bilmes 2012ap.
Once the strong DS decomposition f= f is obtained, the strict subgradients of f at Y C V can be constructed by / h oe E ,y( with hi E 8y (Y), of E 30 (Y), even if f is neither strictly submodular nor supermodular, and the same is true for strict supergradients gy E Hf (Y). This fact enables us to consider submodular- Brregaaanar-Begaman divergences with non sunmouumodular set functions.
Theorem 3.1'. For every set function f R and every Y C V, there exist modular functions by E RN that make (7) and (8) satisfy the conditions in Definition 1.1 respectively.
Breg- In this paper, we call the divergences justified by Theorem 3.1 the cfferrence-ofsuubmoolee
man divergence (DBD) since such divergences no longer need the (strict) submodularity of Gen- erally speaking. finding (strong) DS decomposition = ff' f given f takes exponential com- plexity. One possible approach to benefit from Theorem 3.1² is to prepare the submodular functions f and f beforehand and to construct non-submodullr (Section
3.3 EXPRESSIVE POWER OF DISCRETE BREGMAN DIVERGENCES
We extended the submodular Breegann divergence the DBD which bases a -un-uubmod set function in general. Our question is whether the extension really improves the expressive power as divergence. More generally, does the expressive power of the DBD D, increase when class of the set function f' becomes richer? For example, e-PointNet constitutes superclass of the relaxed facil- ity location functions as described in Subsection 2.3 Clearly, the class of all submodular functions is a subclass of the class of all DS functions, or all set functions.
For precision, we define a class of set functions on 2V as a set of set functions closed under the addition of modular functions.
Definition 3.3. Let F be set of set functions on 2V. We denote the class of the set F as
C(F) = -m
That is, E C(F) implies f + m C(F) for any modular function m E N.N. For example, the set of modular, facility location, submodular, PointNet, and DS functions all induce classes of the corresponding sets. We denote the of all DBDs of functions in C(F) by De(F).
Theorem 3.4. LeiF be sets of set functions and C = C(F), = C(F") be the classes induced by F and F', respectively. IfC C C', then De C De
Proof. Take set function f E C. Suppose for the contrary that a DBD with respect to f' can be
represented by that of another set function f E C. From Dr (X, Y) = D, Y) for all X, Y C V. have D (X, 0) Dy (X, 0) for all X V. Nevertheless, Dy (X, h) is the of fX) and a modular function, and the same for D 0). Hence (X) can be written as the sum of a set function in C (namely f) and modular function, which contradicts f' E C.

---

Published as a conference paper at ICLR 2025
Theorem 3.4 claims that the extension of the uubmodular-Bregman Bregman divergence to the DBD actually increases the expressive power. This motivates us to consider -un-suboo set functions to obtain more capable divergences.
4 PROPOSED METHOD
Following Theorem 3.1' and Theorem 3,4 we propose a capable and learnable divergence that can adapt downstream tasks. Although Theorem 3.2 states the existence of the (strong) DS decompo- sition such that f = fl f² for any f 2 R. the good method to find (strictly) submodular functions f and f is unknown.
To avoid this difficulty, we prepare two submodular NNs a priori and take them as f and f² Given S V. we can find the subgradient of f', hy 0y: and the supergradient of f? o% E a (Y). Then, we measure the dissimilarity between X S V and Y as
Dy(X, (X,Y (D (x,,)
= -xx -M) ( (2(x + ( -x))
(9)
In downstream tasks, the DBD (9) can be learned by some objective function of metric learning, such the triplet loss Hoffer & Ailon 2018).
NUMERICAL EXPERIMENTS
We consider revealing the behavior of the DBD from numerical experiments. architecture, and other details are follows.
The datasets, network
Datasets In our experiments, we use the following datasets.
MNIST (Deng 2012): The MNIST dataset collection of handwritten digits with training set of 60,000 examples and test set of 10,000 examples. Each example is a 28 x 28 image, making a 784-dimensional vector. For an illustrative example (Section 5.1 we use the MNIST dataset to develop discrete problem setting.
ModelNet40 (Wu et al. 2015): The ModelNet40 dataset consists 12,311 meshes in 40 categories (such airplane and chair), of which 9,843 used for training, while the rest 2,468 are reserved for testing. The corresponding point clouds are uniformly sampled from the mesh surfaces. Such point cloud data be regarded therefore suitable for the discrete problem setting that is the focus of this study. We utilize this dataset demonstrate the real-world applicability of our DBD (Section
set of vectors and is
Network Architecture The choice of the NN architecture used as a component of the DBD is arbitrary, long as submodularity guaranteed. In our implementation, we employ s-PointNet (6) with K = 1, the identity function as 7, = 0 or 0.001, and fully-connected multi-layer perceptron (MLP) as Φ. For both ft and f2, the MLPs consist of two hidden layers of 64 units. For the activation functions, the ReLU is used for the hidden and final layers, which yields non-ne nen-tegative outputs; thus the submodularity is guaranteed.
Section 5.2 we also experiment on the DBD without DS decomposition the ablation study. For faimess, the MLP within the model without decomposition adjusted to have x 128 units in the two hidden layers.
Learning Procedure For learning the DBD, we use the triplet loss (Hoffer & Ailon 2018):
C(f) = Σ max (D, -X,
where X XX,X,,X; C V. are i-th anchor, positive, and negative sets, whose construction method will be explained later. We also use Adam (Kingma & Ba 2015) as the optimization algorithm for the gradient update of the NNs, with batch size of 64 and learning rate of 0.001 unless otherwise stated. The extreme point is taken as the subgradient ht (Edmonds, 1970 as in [Iyer & Bilmes, 2012b) and the grow, shrink, and bar supergradients taken gý.

---

Published as a conference paper at ICLR 2025
Query set
Reference sets
DBD
0
2
0.173
0
2
0 0
2
0.309
3
4 1
0.772
1
9
9
0.237
6
9
2
3
9
0.677
3
4
6
1.304
Figure 1: Illustrative example MNIST dataset (Deng, 2012). The query and reference sets constructed from instances the MNIST dataset, and the DBD values correspond to the divergences between the respective set pairs.
5.1 ILLUSTRATIVE EXAMPLE
First, we design experiment to investigate whether our DBD works properly as a divergence. As divergence function, is expected (i) always take non-negative values for any set pair and (ii) take smaller values for similar set pairs and larger values for those that are not. Here, (i) is clearly this subsection, we
ensured by Theorem 3.1 and implementation described Section consider qualitative evaluation using the MNIST dataset check the behavior of (ii). The original MNIST dataset is collection handwritten digit images and we develop a discrete problem setting from this dataset. In this toy experiment, sets of a certain size M. The collection of these sets D is considered as a new discrete version of the dataset, which is used to train the DBD. We set 50,000, M 3, 4, and leam 200 epochs. Now consider the mapping y + y(x) from each instance x E R 784 of the original dataset to its corresponding label yx),, and consider the mapping defined as X {xI,.... XM]]
collect instances in the original dataset generate
(X))
(u(x1)..... ..(......((M))) Given the anchor set X' D, the positive set X and the negative set X are sampled with ensuring the following conditions: X E {x E D v(x)) (XX) + 0) X e
(X E D: (((()) (2(X)= = 0}.
Figure shows the experimental results for this toy example with e For each query set, DBD values between the query and some reference sets are reported. Here, the query and reference sets are arbitrarily chosen so that there are differences in the expected divergence values. For example, the top row of each group of reference sets is chosen to be similar to the query set in terms of the labels of the original dataset, while the bottom row is chosen to be far from it. From this result, we that the DBD takes smaller values for similar set pairs and larger values for those that not.
5.2 APPLICATIONS ON REAL-WORLD DATASET
Next, we confirm the usefulness of our DBD from numerical experiments on the real-world dis- crete dataset. To this end, we consider two applications: set clustering and set retrieval on the ModelNet40 dataset. Here, each instance X E D the dataset is point cloud of size M 500 uniformly sampled from the mesh, and corresponding class label y(X) is given. Since each X E 'D can be regarded as a set size M, the dataset 'D satisfies the discrete prob- lem setting
interested in. Given anchor set X D, the positive and negative sets X X E D y(X)) y(x)) XX (X D ((x)) # y(X) sampled while learning the DBD. In the following, the results of the quantitative evaluation are reported with the means and standard deviations of 10 trials at different random seeds.
8

---

Published as a conference paper at ICLR 2025
Table 2: Experimental results of set clustering for ModelNet40 dataset (Wu al. 2013). Prefixes grow, shrink, and bar correspond to the supergradients (i), ÿy shown in Table MT. The means and standard deviations of 10 trials with different random seeds are reported.
(ï) and ōy (i) respectively (as
Rand index
D)
f(x)
5 0
2 0.001
grow-DBD w/ decomposition
('(X) - (2(X)
0.7942(=0.0092
0.8015(=0.000))
shrink-DBD w/ decomposition
((X) - (x(X)
0.7905 =000088))))
0.7912(=0.0047)
bar-DBD w/ decomposition
((X) - (x()
0.7410(=0.0113
0.7520 =0.0095)
grow-DBD w/o decomposition
((X)
0.7741
0.7743(
(0.0099)
shrink-DBD w/o decomposition
((X)
0.7750
0.7756(=0.005)
bar-DBD w/o decomposition
((X)
N.7303(=0.014))
0.7325(=0.010)
XX YI + JY 1 X AUYYA /Y 1 - (IY) + 2UUD)/2111
X /Y
[X|
0.0225 (00000)
1
0.0232 +0.0059)
1/2
0.0232 +0.0059)
Set Clustering As one of the downstream tasks in the real-world dataset, we consider set clus- tering. In this task, we perform clustering with the k-means algorithm based on the trained DBD, which is a similar setting to the experiments of [yeer & Bilmes (20126). Note that the k-means al- gorithm with Bregman divergence is justified by [Banerjee al. (2005). To investigate the impact of the strict submodularity, which is theoretically required, we conducted experiments with e = (submodular but not strictly submodular) and e 0.001 (strictly submodular), respectively. As an ablation study, the results with a single e-PointNet as f(X) are reported as w/o decomposition, in which f = g. decomposition.
9. Also, w/ decomposition corresponds to the DBD based on the DS
Table 2 shows the experimental results of the set clustering task. For the performance evaluation, we use the Rand index between the resulting clusters and ground truth labels (c.g., airplaine and chair), where the Rand index is a metric used evaluate the similarity between two different clus- terings of the same dataset. The results with some known special cases of the Haanmndulr--Brengne divergence, found in (Iyer & Bilmes, 20126), are also reported. First, the performance impact of the learnalle DBDs is clear because they produce significantly larger Rand index values compared to the submodular Bregman divergences. can be suggested that this is because the exact intersection tends to be almost zero in many cases, each element in the set corresponds to the coordinates of a point cloud. In such cases, the high expressive power and learnability of NNs seem to be very effective. Next, turn our attention to the results of the comparison between the w/ and w/o DS decomposition. These comparisons show that better performance is obtained by the DS decom- position in all three types of supergradients. In addition, looking the standard deviation of the performance evaluation, it can be seen that the DS decomposition slightly reduces the variability of the estimation. We also find that while the grow and shrink supergradients provide comparable performance, the result by the bar supergradient is inferior with statistical significance. As seen in Table the bar supergradient Y, denoted by gy, is independent on Y indeed, meaning that does not use local information around Y unlike the grow and shrink supergradients. The difference may cause the performance gap of the supergradients. When striet submodularity is introduced with 0.001, we find a slight performance improvement compared to = O, though it is not statisti- cally significant, and the variance is also reduced. In addition, we report the results with the Softplus activation function in Appendix B
Set Retrieval Finally, we consider a set retrieval task for the qualitative evaluation of the DBD. The set retrieval is a task similar to the image retrieval task, where each instance is a set. In this task, we choose an arbitrary query set XQ E D. For this query set XQ. we seek top-K similar sets (X...., Xx) the resulting top-K sets should be qualitatively similar to the query set. In this experiment, we set K and query sets are chosen from the airplane and chair classes. Figure[ishowe the experimen- tal results of set retrieval with = O. The leftmost column shows the query set and the right columns the corresponding top-5 similar sets. In the visualization of point cloud data, the rotation of the
that minimizes Dy Xx). If the DBD is learned as reasonable divergence,

---

Published as a conference paper at ICLR 2025
Query set
Top-5 retrieval results
Figure 2: Experimental results of set retrieval for ModelNet40 dataset (Wu al. 2015). For each given query set, retrieved top-5 similar sets are listed based on the DBD.
camera is arbitrary, but we rotate it for the sake of clarity so that the class to which it belongs is easily recognizable. addition, since our DBD quantifies the differences between discrete data, it is naturally invariant to these rotations. From the results of this experiment, shown in Fig, [ it can be seen that the DBD is capable of retrieving similar sets. particular, the query set and the corresponding similar sets are consistent in the classes to which they belong, indicating that the learning of the DBD is achieved as expected. Also, although omitted this figure, the divergence between identical query sets Dy XQ) satisfies D (Xq, XQ) < DD Xx), VI < S and behaves as expected of the divergence, taking the smallest value between identical instances. Additionally, we show the quantitative score the set retrieval experiment in Table 4 (in Appendix B). Despite using only simple MLP architecture without any pretraining, our method closely ap- proaches the state-of-the-art method (Hamdi al., (02T) and achieves better performance than its previous method (Liu et al. 2019). This clearly demonstrates the capability of the DBDs
CONCLUSION AND DISCUSSION
Traditional Bregman divergences are defined over continuous spaces using convex functions. In this study, we extended this concept to discrete finite sets by proposing a difference-of-suumodii Bregman divergence. This extension enables natural handling of data structures such point clouds and item sets.
While previous approaches have used submodular functions-analogenn to convex functions-46 define submodular Bregman divergences on discrete sets, we showed that liffereence-f-suubooole Bregman divergences can be defined from any set function, not necessarily submodular, by utilizing difference-of-submodle
decomposition. Theoretically, we showed that enhancing the expressive power of the set function leads
expressive differenceeof-summodull Bregman divergence. This finding motivates the construction of cawerrecceeo--uummdooeee Bregman divergences by learn- ing flexible functions for specific tasks.
To achieve this, we proposed an approach that learns set functions for differennee-o--submoduu Bregman divergences using permutation-invariaa imentally, we confirmed that the difference-on cnfference-oo-suumolle of-muuuuull Bregman divergence generated from set function defined the difference of set functions represented by c-PointNet is a proper diver- Bregman
neural networks, particularly e-PointNet. Exper-
gence. Furthermore, in clustering task, the PointNet-based differencee-f-submoll divergence- learned from small amount of ground-truth cluster data data- significantly outperformed existing submodular Bregman divergences constructed on fixed submodular functions.
For future work, the exploration of architectures for the permutation invariant NNs is an impor- tant task. We used the simple e-PointNet architecture in the numerical experiments, but optimal hyperparameters (e.g., number of units and layers, dimension K, and E) were not discussed. Fur- thermore, novel flexible NN architectures that model submodular functions proposed (Bilmes & Bai 2017 Bhatt et al. 2024). From an engineering perspective, it is important to develop a method that performs well better than the state-of-the-arr methods according to our framework.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
We thank anonymous reviewers for insightful comments and suggestions. Part of this work is sup- ported by JST CREST Grant Number JPMICR2015, JST PRESTO Grant Number JPMJPR24K5, JST the establishment of university fellowships towards the creation of science technology innova- tion Grant Number JPMJFS2136, and JSPS KAKENHI Grant Numbers JP22H03653, 19K20212 and 24K21315.
REPRODUCIBILITY
We provide the code needed to reproduce experiments in the supplementary material attached. The proofs omitted from the main are presented in Appendix A
REFERENCES
Shun-ichi Amari. Information Geometry and Its Applications. edition, 2016.
Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, and Joydeep Ghosh. Clustering with Breg- man Divergences. Journal of Machine Learning Research, 6(58): 7705-179, 2005.
Amir Beck and Marc Teboulle. Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization. Operations Research Letters, 31(3):167-175, 2003.
Gantavya Bhatt, Arnav Mohanty Das, and Jeff Bilmes. Deep submodular peripteral networks. In The Tiirt Annual Conference on Neural Information Processing Systems, 2024.
Jeffrey Bilmes and Wenruo Bai. Deep Submodular Functions, 2017.
David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational Inference: A Review for Statis- ticians. Journal of the American Statistical Association, 12(518):859-877 2017.
Seung-Seok Choi, Sung-Hyuk Cha, Charles C Tappert, et al. A survey of binary similarity and distance measures. Journal of systemics, cybernetics and informatics, 8(1):43-48, 2010.
Andrzej Cichocki and Shun-ichi Amari. Families of Alpha- Beta- and Gamma- Divergences: Flex- ible and Robust Mcasures of Similarities. Entropy, 1236):153 2010.
Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S Dhillon. Information-theoredt metric learning. In Proceedings of the 24th International Conference on Machine Learning, pp- 209-216. ACM, 2007.
Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6): 2012.
Jack Edmonds. Submodular Functions, Matroids and Certain Polyhedra. Combinatorial Structures and Their Applications, 1970.
Oisin Faust, Hamza Fawzi, and James Saunderson. A bregman divergence view on the difference- of-convex algorithm. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent (eds.), Pro- ceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 3427-3439. PMLR, 25-27 Apr 2023.
Satoru Fujishige. Submodular Functions and Optimization. 2005.
Jacob Goldberger, Sam Roweis, Geoffrey Hinton, and Ruslan Salakhutdinov. Neighbourhood com- ponents analysis. In Advances in Neural Information Processing Systems, volume 17, pp. 513- 520. MIT Press, 2005.
Abdullah Hamdi, Silvio Giancola, and Bemard Ghanem. MVTN: Multi-view transformation net- work for 3d shape recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1-11, 2021.
Elad Hoffer and Nir Ailon. Deep metric learning using Triplet network, 2018.

---

Published as a conference paper at ICLR 2025
Rishabh lyer and Jeff Bilmes. Algorithms for approximate minimization of the difference between submodular functions, with applications. In Proceedings of the Twenty-Eige Conference on Uncertainty in Artificial Intelligence, UAI 12, pp. 407-417, 2012a.
Rishabh Iyer and Jeff A Bilmes. Gubmodular-Bregmaen and the Lovász-Bregman Divergences with Applications. In Advances in Neural Information Processing Systems, volume 25, 2012b.
Rishabh Iyer, Stefanie Jegelka, and Jeff Bilmes. Fast semidifferential-basee submodular function optimization. In Proceedings of the 30th International Conference on International Conference on Machine Learning Volume 28, ICML' 13, 2013.
Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General Perception with Iterative Attention. Proceedings of the 38th International Conference Machine Learning, pp- 4p.5665144555 (661-4664, 2021.
Jinwoo Kim, Jaehoon Yoo, Juho Lee, and Seunghoon Hong. SetVAE: Learning Hierarchical Com- position for Generative Modeling of Set-Structured Data. In Proceedings of the IEEE/CV Con- ference on Computer Vision and Pattern Recognition, pp. 15059-15068, 2021.
Masanari Kimura, Ryotaro Shimizu, Yuki Hirakawa, Ryosuke Goto, and Yuki Saito. On permutation-invariant neural networks, 2024.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega CA, USA, 2015.
Brian Kulis, Sugato Basu, Inderjit S Dhillon, and Raymond Mooney. Learning a mahalanobis distance metrie for clustering via information maximization. In Proceedings of the 22nd Interna- tional Conference on Machine Learning, pp. 481 481-488. ACM, 2005.
S. Kullback and R. A. Leibler On Information and Sufficiency. The Annals of Mathematical Statistics, 729-86, 1951.
Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set Transformer: A Framework for Attention- based Permutation- Invariant Neural Networks. In Pro- ceedings of the 36th International Conference on Machine Learning, pp. 3744-3753, 2019.
Xiang Li and H. George Du. A short proof for stronger version of DS decomposition in set function optimization. Journal of Combinatorial Optimization, 40(4))901-906, 2020.
Zhiyuan Li, Ziru Liu, Anna Zou, and Anca L. Ralescu. Learning empirical bregman divergence for uncertain distance representation. In 2023 26th International Conference on Information Fusion (FUSION), volume pp. 1-8. IEEE, June 2023.
Yongcheng Liu, Bin Fan, Gaofeng Meng. Jiwen Lu, Shiming Xiang, and Chunhong Pan. Dense- point: Learning densely contextual representation for efficient point cloud processing. In Pro- ceedings of the IEEF/CVF international conference on computer vision, pp. 5239-5248, 2019.
Fred Lu, Edward Raff, and Francis Ferraro. Neural bregman divergences for distance learning. In The Eleventh International Conference on Learning Representations, 2023.
Renqiang Min, David A. Stanley, Zineng Yuan, Anthony Bonner, and Zhaolei Zhang. deep linear feature mapping for targe-margin knn classification. In 2009 Ninth IEEE International Conference on Data Mining, pp. 357-366, 2009.
Mukund Narasimhan and Jeff Bilmes. A coimmodular-uppemmodoe procedure with applications to discriminative structure learning. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, UAI'05. pp. 404 412, 2005.
G. L. Nemhauser, I. A. Wolsey, and M. I.. Fisher. An analysis of approximations for maximizing submodular set functions-I. Mathematical Programming, 14(1):265-29) 1978.
Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652-660, 2017.
12

---

Published as a conference paper at ICLR 2025
Mina Rezaci, Farzin Soleymani. Bernd Bischl, and Shekoofeh Azizi. Deep bregman divergence for self--uper representations learning. Computer Vision and Image Understanding, 235: 103801, 2023. ISSN 1077-3142.
Ruslan Salakhutdinov and Geoff Hinton. Learning a nonlinear embedding by preserving class neigh- bourhood structure. In Marina Meila and Xiaotong Shen (eds.), Proceedings of the Eleventh Inter- national Conference on Artificial Intelligence and Statisties, volume of Proceedings of Machine Learning Research, pp. 412-419 San Juan, Puerto Rico, 21 21-24 Mar 2007. PMLR.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815-823. IEEE, 2015.
Ali Siahkamari, XIDE XIA, Venkatesh Saligrama, David Castañón, and Brian Kulis. Learning to approximate a bregman divergence. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 3603-3612.
Curran Associates, Inc., 2020.
Hyun Oh Song. Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 4004-4012. IEEE, 2016.
Peter Stobbe. Convex Analysis for Minimizing and Learning Submodular Set Functions. PhD thesis, California Institute of Technology, 2013.
Martin J. Wainwright and Michael I. Jordan. Graphical Models, Exponential Families, and Varia- tional Inference. Foundations and Trends in Machine Learning, 1(1-2):1-305, 2008
Fei Wang and Jimeng Sun. Survey on distance metric learning and dimensionality reduction in data mining. Data Mining and Knowledge Discovery, 29:534-566 2014.
Jiang Wang. Feng Zhou, Shenghua Wen, Xiao Liu, and Yuanqing Lin. Learning fine aineggrined image similarity with deep ranking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1386-1393. IEEE, 2014.
Kilian Q Weinberger, John Blitzer, and Lawrence Saul. Distance metric learning for large margin nearest neighbor classification. In Y. Weiss, B. Schölkopf, and J. Platt (eds.), Advances in Neural Information Processing Systems, volume 18. MIT Press, 2005.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Jiahao Xie and Guangmo Tong. Advances in set function learning: A survey of techniques and applications. ACM Computing Surveys, 2025.
Eric Xing, Michael Jordan, Stuart J Russell, and Andrew Ng. Distance metric learning with ap- plication to clustering with side-information. In S. Becker, S. Thrun, and K. Obermayer (eds.), Advances in Neural Information Processing Systems, volume 15. MIT Press, 2002.
Han-Jia Ye, De chuan Zhan, and Yuan Jiang. Fast generalization rates for distance metric learning.
Machine Learning. 108:267 295, 2018.
Jieping Ye, Zheng Zhao, and Mingrui Wu. Adaptive distance metrie learning for clustering. In Proceedings of the 20th International Conference on Neural Information Processing Systems, pp- 543-550. Curran Associates Inc., 2007.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep Sets. In Advances in Neural Information Processing Systems, vol- ume 30, 2017.
13

---

Published as a conference paper at ICLR 2025
A PROOFS
A.1
WELL-DEFINERE OF STRICT SEMIDITERENTIALS
In this section, we show the well-definedness of strict semidifferentials defined in and (5) First, confirm the equivalence between the strict submodularity and the strict diminished turns property as in (Nemhauser et al., 1978). Hereafter, we denote the increase of set function (ī)) f(S) for V and S VMJ} and distinguish from by whether contains an equality or not. We repeat the definition of strict submodularity for the sake of convenience:
as S)
Definition A.1. A set function : - R is strictly submodular if
f(X) + f(Y) ((XUU)+((XN))
(10)
for every comparabe X.Y.V.
Lemma A.2. For a set function f: 2V + R. the strict submodularity 10
holds if and only if
fGls) > f((z)
(11)
satisfied for all C C C and j VVT.
Proof.
[ 1T
By taking S CII we have
[T0p] Let X,Y C V be any non-comparable subsets of V and denote X = f.i,,..... # 0. From
[[]]
JilXnY (i,, (i. (1-))) JUilY (i1...
holds. By summing up the above equality, the l.h.s. becomes
fulxnru - = Ei (t.. 1=1
= (AM
and the r.h.s. becomes
f (f-))...f-11))- u ( - f
=
That results strict submodularity (10..
Lemma states the strict version the diminishing returns property with respect to strictly sub- modular functions. We can find the following proposition from Lemma A.2
Lemma A.3. When f 2 - IR is strictly submodular,
F(T)-(()) < Σ f(is)- Σ FG(SUT(()))
(12)
JET'S
JESYT
and
S(T) -(S) Σ SUiSTT) Σ SUSS)()
(13)
JET'S
JESYT
for every S, V (S+
14

---

Published as a conference paper at ICLR 2025
Proof. We strict diminishing returns 11 to prove (2).. Taking any S,T S V,, (S + T) and denoting T,S = (i))....Jr), S = {ki...... /.g. We have
- - Eu(su ((SU..
Σ ( (A.........)1)1)))
Σ SGiSs) E fG1s)
(14)
JET
from 11 (the equality holds only if S = 0). Then,
9 ((SUT) 2 [T(T (......., ko) f(T k(-1))
Σ [U (k1,.......1)).
[UNITU (oMMk))
Σ ((A)) E SGISUT(A)) JESTT
(15)
is also obtained (the equality holds only S = 0). By subtracting (15) from (4 we can confirm
Note that the inequality in (12) does not include an equality unlike and because at least one of T and T is non-empty by the condition S * T.
12p.
Inequality
13 can be derived in a similar manner. We can obtain
(( - ((S =
JollssT)
Σ
f(ISnT) = Σ f(iSST)
(16)
JETVS
and
f(S)=) (((SnT) f((SNT) U (k....., k-11))
Σ ((KISS(A))))) Σ SG(S()))
(17)
JES(T
where the equalities [6 and (7] hold only if T = and S 0. respectively. Inequality
13 is obtained from [6) and 17
15

---

Published as a conference paper at ICLR 2025
We may also be able show the sufficiency of (12) and 13) for the strict submodularity (0) in
similar way to (Nemhauser et al. 1978 Proposition 2.1), but we omit because it is not necessary for our methodology.
Now we show the existence of strictly subgradients of strict submodular functions.
Proposition A.4 (Existence of strict subgradients). Suppose that f : 2 + R satisfies the strict submodularity 2 Then, the strict subdifferential of f Y C V non-empty.
Proof. We show the existence by directly composing specific examples of the subgradients as in (Stobee 2013 Section 3.3.1). Let À IR N be some modular function and E be some real value. We define a modular function hy R as
by = x= #=>((0y--ux)) Inx)..
For any X S V, can find
(x) by (Y) - (by, 1x -xxxx)) - - lv)
and the rightmost term can be rewritten as
(Iy Ivy, 1x lv) 2 1x - 1v) (Ir + Ivx, lv)
-(XXXY)-2)XXX))
Since the r.h.s. is the negative of the size of the set of X and Y XORed, it takes negative value when + Y. Therefore, we can have
vX 2 (Y), by (X) by (Y) (A, - ly) t(ly 1x 1 <(x) f(Y)
by taking sufficiently large > O. To be precise, we obtain
f(Y) - f())+ MXX
by = A - lv + t>
max (Y2
Next, let us prove the con-emmtiness of strict superdifferentiaas.
Proposition 2.5 (Strict supergradients). The modular functions gy gy, and ğy defined in Table are all the strict supergradients if f is strictly supermodular.
Proof. For all X C V, we have
gr (x(X) Σ dr(1)= Σ 1GIV) Σ SOIVA)).
JEX
jEXiY
JEXNY
(Y)
Σ (r)
Σ SGIVMGH) E SGIVA)),
JEY
JEY(X
JEXnY
by definition. If + Y, it can be found that
(x) -y (Y) = Σ f(lM) Σ SUlViSH
JEX(Y
Σ 101V) Σ sOXUU(( (: Lemma A.2
JEX(Y
f(X) fYY) (: Lemma
16

---

Published as a conference paper at ICLR 2025
holds. This indicates ÿy (Y). Similarly, have
-x = Σ f(10) E FGIVMD
JEX
jEY
Σ f((10)
E SUIYMA) JEY
Lemmm A.2)
JEXVY
= (x -(x)))rr())
Σ f((XnY) E fUIY(U) (: Lemma A.2
JEXVY
jEY
[(X) - I(Y)
Le- A.3
for X + Y. Thus .y E 0 (Y) is also obtained.
A.2 STRICT SUBMODULARITY OF THE RELAXED FACILITY LOCATION FUNCTION
Proposition A.5. The relaxed facility locution fanction JFC,e defined in (3 is strictly submodular for O.
Proof. For every X c and every VX,, we have
K
fic,) = Σ
log Σ
ebix/e +
teosule
-
log
eba/E
k=1
VEX
Ex
eBsa/s
log
1+
(18)
EiEx
edu/e
1=1
Now, consider X, Y V such that X CY C and j E VYY Then,
ensale
fFC, =
log 1+
Σiex eduiE
emsk/E
> log
1+
Eiey edil/
k=1
= fFc..(jll))
holds from 18 By Lemma
A.2
the strict submodulariyy of fro.c is shown.
B ADDITIONAL EXPERIMENTS
B.1 ANOTHER ACTIVATION FUNCTION
Additionally, report the results of experiments where the activation function in the final layer of s-PointNet's MLP was changed from ReLU to Softplus. Table 3 shows the results with s 0. No significant difference was observed compared the results obtained with ReLU, as reported in
Table 2
B.2 QUANTITATIVE EVALUATION FOR SET RETRIEVAL
Table shows the result of the quantitative score of the set retrieval experiment in Section Retrieval mean Average Precision (mAP) is employed the metric in (Hamdi et al.) 2021). We compare the performance of the proposed DBD with PointNet (see Section [5) for the architecture) with baseline methods, Densepoint (Liu et al. 2019) and the multi-view transformation network (MVTN) (Hamdi et al. 2021). For MVTN, we report the score shown in the original paper (Hamdi al. 2021 because failed reproduce the result.
5.2
17

---

Published as a conference paper at ICLR 2025
Table 3: Experimental results of set clustering for ModelNet40 dataset (Wu et al. 2015) with the softplus activation function. Prefixes grow, shrink, and bar correspond to the supergradients gy (j), ÿy (j) and ōy (j) respectively (as shown in Table D. The means and standard deviations of 10 trials with different random seeds are reported.
D)
f(x)
Rand index
grow-DBD (softplus) w/ decomposition
((X) F(()
0.7940((10.088)
shrink-DBD (softplus) w/ decomposition
f(X) - F2(X
0.7911 (1.001(+0.0090)
bar-DBD (softplus) w/ decomposition
('X)- (x()
0.7424(+0.0107)
grow-DBD (softplus) w/o decomposition
(())
0.7743(#0.0095)
shrink-DBD (softplus) w/o decomposition
P(X)
0.7756(10.0094)
bar-DBD (softplus) w/o decomposition
('(X
0.7295(10.0120)
Table 4: Experimental results of set retrieval for ModelNet40 dataset (Wu et al. 2015). Prefixes grow, shrink, and bar correspond to the supergradients ÿy (j), ÿy (j) and gy ()) respectively (as shown in Table D. The means and standard deviations of 10 trials with different random seeds reported, except =, Note that the evaluation result with * is borrowed from the original paper.
Method
mAP
grow-DBs DBD w/ decomposition
90.13(+0.75)
shrink-DBD w/ decomposition
90.20(+0.77
bar-DBD w/ decomposition
86.09(
grow-DBD w/o decomposition
88.12 -0.80)
shrink-DBD w/o decomposition
88.20 =0.81)
bar-DBD w/o decomposition
83.57 +0.97)
Densepoint (Liu et al. 2019)
89.68 10.88)
MVTN (Hamdi et 2021
92.9"
18


Reviews:
[{"review_id": "L8OtyrUCZG", "reviewer": "Reviewer_d1i9", "summary": "Review of \"DISCRETE BREGMAN DIVERGENCE\" submitted to ICLR 2025.\n\nThis paper extends the submodular Bregman divergence methods introduced a few years ago to non-submodular functions. They do this via the notion of a DS (difference of submodular) decomposition of non-submodular functions. The key thing is that in order for the identifiability property of the Bregman divergence to hold (i.e., that D(x,y) = 0 iff x==y), they note that the submodular function needs to be strict (i.e., lie on the interior of the submodular cone), something they point out was not mentioned in the past (although arguably it was implicit). They note that any set function has a DS decomposition in terms of two strict submodular (or two strict supermodular) functions and since submodular functions have both sub-gradients and super-gradients, it is possible to use the two DS components of any set function to define a sub-gradient and a super-gradient based Bregman divergence, and therefore define a Bregman divergence for any set function.\n\nThey go on to show that these functions can be learnt, i.e., one can learn two submodular functions and then find the semi-gradients of these to produce a discrete Bregman divergence based on these learnt submodular functions. They show results on some set clustering that seem to me to be reasonable.\n\nWhile I do not think that the paper is revolutionary, and I do think that the strictness of previous DS decompositions was implicit, I do think it is worth pointing that out more explicitly as this paper (and also the Li & Du, 2020) in fact do, so I agree with this. There are a few issues of tone, however, that I would change in the paper and that I point out below. Also there are a few recent citations that I think you should add. All in all, however, I think the paper should be accepted as it is a nice contribution, and it is in particular good to see the empirical work using their submodular-supermodular Bregman divergence methods.\n\nHere are some comments.\n\nFirstly, I think you might consider changing the title of the paper a Submodular-Supermodular Bregman Divergence, or \"Discrete DS Bregman Divergence\" since the approach is entirely dependent on there being a DS decomposition. If one only has oracle access to a non-submodular non-supermodular set functions, it can be hard to find a reasonable decomposition (assuming one knows bounds of the function, one can always add and subtract a very large strict submodular function to any set function to transform it to a DS function but that is a fairly vacuous DS decomposition). So unless you really can produce a Bregman Divergence for any set function given only oracle access, I think it is more appropriate to entitle your paper \"Submodular-Supermodular Bregman Divergence\".\n\nI think you may want to change the tone of lines 213-216 where you say \"a formal discussion on the well-definedness is lacked\" as that sounds a bit disparaging. You are basing your results strongly on their methods, standing on their shoulders, so you might say something along the lines of \"This earlier work, however, did not explicitly mention that in order for the identifiability property of the Bregman divergence to always hold, it is necessary for the submodular functions involved to be strict\", i.e., be more explicit in what you are building on rather than saying that the previous paper \"is lacked\".\n\nI think the numerical experiments are good and, as mentioned above, it is good to see empirical work as well on submodularity, I think there should be more such things.\n\nThe submodular functions that you are learning however seem to be either deep submodular functions (DSFs), i.e., see (Bilmes & Bai from 2017, https://arxiv.org/abs/1701.08939) or much more recently deep submodular peripteral networks (DSPNs, https://arxiv.org/abs/2403.08199 from 2024). I think both papers should be cited. In particular, it seems your submodular functions are simple forms of DSPNs, but I think that one could learn two DSPNs and construct one of your Bregman divergences from semi-gradients of DSPNs quite easily, and this would both further extend the expressivity of your Bregman divergences and also extend the utility of these DSPNs. Also DSPNs strictly extend DSFs (removing the only known limitation of DSFs), and this is also useful for discrete Bregman divergences.", "questions": "see above.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "see above.", "weaknesses": "see above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zSwTneI1CO", "reviewer": "Reviewer_VMyv", "summary": "The submission proposes new class of divergences on discrete spaces and a learning framework comprised of permutation-invariant neural networks with metric learning loss.\n\nLeveraging difference-of-submodular (DS) decomposition for any set function f, the authors obtain more expressive class of Bregman divergences dubbed discrete Bregman divergences (DBD). Expressiveness advantage over submodular Bregman divergences is achieved by extending the underlying set function class to not necessarily be submodular but rather admit DS decomposition.\n\nThe paper validates the proposed approach to learning DBD in a set of numerical experiments.", "questions": "Formally, the proposed implementation is not divergence as per the Definition 1.1. But rather the generalized divergence? What are the consequences of that in practice? In what scenarios the difference may play a crucial role?\n\nAs noted in the weaknesses section, the expressiveness of the new DBD is fully explored empirically. Can we quantify how much more expressive the new class of divergences is compared to the submodular Bregman divergences? I think this is important to show the strength of the new DBD and should be explored empirically.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper addresses a problem of extending Bregman divergences to discrete spaces. This is an important problem as many methods that rely on Bregman divergences in continuous settings may be adapted to discrete scenarios.\n\nThe submission does a good job providing a structured and clear background on the treated problem.\n\nThe central idea of the paper stems from DS decomposition. Typically, one needs a submodular function $f$ to instantiate submoduler Bregman divergence. However, the authors notice that DS decomposition proposed in prior work alleviates this constraint on $f$ itself, and propose to use the difference of two submodular functions, $f = f^1 - f^2$. I'm not an expert in the area, but it seems the idea of using DS decomposition in forming more expressive class of Bregman divergences on discrete spaces has not been explored in the literature and is novel.\n\nThe authors provide empirical validation for the proposed learning framework. Given two submodular $f^1, f^2 $ functions, DS decomposition facilitates discrete Bregman divergence specification and with proper implementation of $f^1, f^2$ one can utilize metric learning approach to learn divergence from data. This defines a natural combination of permutation-invariant neural networks (to work with sets) and triplet loss (to learn the divergence). With MNIST experiment, they illustrate that the learned divergence provides reasonable results for dissimilarity between sets of images, with similar image sets getting smaller dissimilarity scores. Point cloud dataset experiments validate that the use of novel divergence class provides benefits over the use of submodular Bregman divergences in set clustering experiment. They further validate that the divergence learned provides semantically close set retrieval.", "weaknesses": "At the very start, the paper highlights the problem of identifiability of divergence, which comes from divergence definition $D(x, y) = 0 \\rightarrow x=y$. Thus, later the submission puts a lot of attention on the strict submodularity of the underlying set functions in Bregman divergences as one needs strict inequalities to satisfy the definition requirement. However, the proposed implementation seems problematic to me in this sense. Indeed, the submodularity is guaranteed, but DBD requires strict DS decomposition, so $f^i$s should be strictly submodular, which doesn't seem the case for the adopted architecture:\n$$\nf_{PN}([x_i]_{i \\in X}) = \\max_{i \\in X} ReLU(h(x_i)),\n$$\nwhere $h: \\mathcal{X} \\rightarrow \\mathbb{R}$ instantiated with an MLP with last activation set to ReLU. When argmax is achieved outside the intersection, the inequality from Definition 2.3 holds, but if argmax is inside the intersection, we fail to meet strict modularity. So the proposed implementation doesn't match the requirements of DBD. Perhaps I'm missing something here?\n\nSince the proposed approach targets practical side of things, it seems it needs more extensive experimentation. For example, one would expect to see expressiveness comparison that is not limited to only one task (set clustering) and one dataset (ModelNet40). This will help gain more empirical evidence for the substantial gain in expressiveness across tasks and task complexities, and justify the use of DBD which requires more computational resources (two functions instead of one).\n\nThe authors didn't discuss the limitations of their approach, which partly stems from limited experimentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QpgHxM22iP", "reviewer": "Reviewer_QZP6", "summary": "In this work, the authors generalize discrete Bregman divergences (DBDs) introduced in Iyer and Bilmes (2012a) to the case where the generating functions are not restricted to be submodulars. By leveraging the fact that any set function can be decomposed as the difference of two submodular ones, the authors show that any set function induces a DBD, and that this extension enables to define larger classes of DBDs. Additionally, they propose a framework to learn such divergences from observations by leveraging existing permutation invariant architectures, such as PointNet, that are by construction submodulars. While obtaining the decomposition of a set function as a difference of two submodular ones take exponential complexity, the authors propose to directly model generalized DBDs using the difference of two parametrized submodular functions obtained from PointNets. Then, they propose to learn an adapted DBD from labeled observations using the triplet loss, and show experimentally the application of their approach for clustering and retrieval tasks on two real-world datasets.", "questions": "How does the proposed approach compare with other baselines for clustering and retrieval tasks?\nCan the authors provide more experimental evidence on the advantage of considering their generalized DBD rather than using the standard one with submodular generating functions?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and presented in a clear, accessible manner. The authors thoroughly acknowledge the relevant literature and prior work, thereby enhancing the clarity of their contributions. To the best of my knowledge, the proposed generalization of DBDs is novel, as is the framework introduced for learning them.", "weaknesses": "The main weaknesses of the paper are twofold: (1) a lack of motivation, and, relatedly, (2) a lack of empirical evidence to demonstrate the benefits of the proposed approach. Regarding the first point, while the paper introduces a new mathematical tool—the (generalized) DBD—the motivation for its introduction is not sufficiently developed. Although the authors demonstrate that their extension allows the definition of larger classes of DBDs, the motivations for considering DBDs in the first place for ML tasks are not clearly articulated. This should be clarified in the introduction and related work. Concerning the second point, the authors aim to demonstrate the applications of this tool for clustering and retrieval tasks; however, the experimental results might be insufficient to demonstrate how the proposed generalization improves upon standard DBDs using only submodular generating functions, as only one experiment is provided to demonstrate this point. Furthermore, it remains unclear how DBDs compare to other baseline methods capable of addressing similar tasks as no comparative analysis has been carried out. To enhance the impact of their work, I suggest that the authors strengthen the motivation, and compare the proposed approach with SoTA baselines on the tasks considered.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JsvdYjq7zw", "reviewer": "Reviewer_JSTp", "summary": "Bregman divergence is a pseudo-distance for comparing vectors or functions. In this paper, the authors present a new technique to construct such divergence on a discrete domain. Specifically:\n\n1. The authors prove that a strictly submodular function can induce a Bregman divergence (Theorem 3.1).\n2. They extend this property, showing that when the function has the form $f + m$, where $m$ is a modular function and $f$ is neither submodular nor supermodular, it can also induce a Bregman divergence (Theorem 3.1').\n3. Finally, they demonstrate that the broader the function class, the broader the class of induced divergences (Theorem 3.4).\n4. They provide a numerical form of the proposed discrete Bregman divergence (Section 4).\n5. Numerical experiments show that the learnable discrete Bregman divergences can capture structure and outperform existing methods in downstream tasks.", "questions": "1. Regarding the notation in Theorem 3.1, it appears that $h_Y$ and $g_Y$ are indeed independent of the set $Y$. Is that correct?\n\n2. In Table 2, could you clarify the values in each column? Specifically, are they the mean and variance of 10 trials of what particular measure?\n\n3. In Theorem 3.1' and the contribution section (lines 069-071), the authors mention proving that $f$ does not need to be submodular. However, in Section 4, lines 306-307, it appears that the implementation still requires submodular functions $f_1$ and $f_2$. What, then, are the advantages and differences of the new divergence construction technique compared to the original one by [Iyer and Bilmes (2012a)], which requires submodularity? It seems that the proposed technique still relies on submodular properties in the implementation stage.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The authors extend previous work on constructing Bregman divergence by relaxing the requirement that the generator function $f$ must be submodular.  \n2. They present a numerical form of the proposed discrete Bregman divergence using permutation-invariant neural networks.  \n3. In the experiment section, the authors demonstrate that the constructed Bregman divergence returns smaller values for similar set pairs and larger values for dissimilar set pairs.", "weaknesses": "1. In Section 5.2, *Real Data Application*, it appears that there are no baselines for either the clustering or shape retrieval experiments. Figure 2 demonstrates the performance of the proposed Bregman divergence, but it is difficult to see how this new divergence improves upon previous divergences, such as those presented in [Iyer and Bilmes (2012a)].\n\nAdditionally, quantitative metrics (e.g., accuracy in the shape retrieval/classification experiment) and wall-clock comparisons to assess the performance of the proposed divergence should be included and discussed.\n\n2. The differences between the new divergence construction technique and previous work [Iyer and Bilmes (2012a)] remain unclear. For instance, in Equation (7), based on Iyer’s work, $f$ should be submodular, and $h_Y$ serves as its subgradient, which is straightforward to construct. \n\nIn the new divergence construction technique proposed here (presumably based on Theorem 3.1'), $f$ can be any set function, which raises the challenge of finding the appropriate modular mapping $h_Y$. In short, the old technique imposes a requirement on $f$, making $h_Y$ straightforward once $f$ is constructed. The new technique has no requirement for $f$ but requires finding an appropriate $h_Y$. Given this, it is not immediately clear why the new technique would outperform the previous one.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OSVCuOW7Ed", "reviewer": "Reviewer_J6xx", "summary": "The authors propose a novel method to construct Bregman divergences for functions defined on discrete sets. Their method is backed by new theory, and it is learnable and applicable to deep-learning on set data.\n\nTheir main contributions are:\n1. They provide theoretical justification for the technique of [1] to construct Bregman divergences from strictly submodular generator functions. Namely, they prove that the induced divergence is indeed a divergence.\n2. They provide a technique to construct Bregman divergences from *arbitrary* generator set-functions, which need not be submodular. They do so using a submodular analogue of the Difference-of-Convex decomposition.\n3. They motivate their construction theoretically by proving that a larger class of generator functions makes for a larger class of induced Bergman divergences.\n4. They demonstrate the applicability of their method to deep learning via preliminary experiments. In their experiments they use a neural network based on their construction, which computes discrete Bergman divergences that are learnable.\n   - They train their architecture to comptue divergences between sets of MNIST digits, and show through examples that the learned metric makes sense.\n   - They demonstrate the advantage of their construction compared to simpler ones (based solely on modular set-functions, or on basic set operations) by performing k-Means clustering on PointNet-40, treating the point clouds as sets and using the computed divergences as a distance measure.\n   - They further demonstrate their method in the task of set-retrieval on PointNet-40, showing in several examples that the top 5 retrieved examples indeed belong to the correct class.\n\n[1] Faust, Fauzi, Saunderson (2023) - A Bregman Divergence View on the Difference-of-Convex Algorithm", "questions": "## Major comments\n\n1. Crucial details regarding your experiments are missing. For example:\n   - Line 359: With that loss function, it seems that the global optimum of zero is trivially attainable. What prevents your method from collapsing to zero?\n   - What architectures did you use in practice? (dimensions, numbers of layers etc.)\n   - Line 307: How did you compute the subgradients and supergradients of the neural networks a priori?\n   \n   The paper could seriously benefit from an additional appendix describing the technical details of your experiments.\n2. Line 458: \"In addition, since our DBD quantifies the differences between discrete data, it is naturally invariant to these rotations. Note that the rotational invariance in point cloud data corresponds to the permutation invariance in set data and this property is not provided by usual divergences over vectors.\"\n   This argument and reasoning are unclear. Are you saying that the DBD between two point-clouds X,Y is rotation invariant, namely D(X,Y) = D(RX,RY) for all rotations R? If this is the case, it is highly nontrivial and requires proof. Anyway the argument itself requires clarification.\n3. Line 20: \"outperforming existing methods on tasks such as clustering\" is too strong a statement, as you did not compare with existing methods outside the realm of discrete set functions.\n4. The continuous analog of the core idea in this work, namely to construct Bregman divergences from arbitrary nonconvex generator functions using the Difference-of-Convex decomposition, was studied in [1]. Their contribution should be acknowledged.\n\n## Minor comments\n\n1. How come bar-DBD with decomposition performed worse than grow- and shrink-DBD without decomposition? Is there an intuitive explanation? Are there any examples where you observed a stronger benefit to non-modular over modular generator functions? This could provide stronger empirical support for your theoretical contribution.\n2. Lines 422-424: For clarity, I suggest stating explicitly that you calculated the Rand index between the resulting clustering and the clustering induced by the ground-truth labels.\n3. In l.213-218, I suggest replacing \"a formal discussion on the well-definedness is lacked\", which sounds vague, with a clear and explicit statement of what you prove that the referred paper did not.\n\n## Possible errata\n\n1. Lines 213-218 and 236: The citation (Iyer & Bilmes 2012a) should probably be (Iyer & Bilmes 2012b).\n2. Line 447, the first instance of 'bar' should probably be 'shrink'.\n3. Line 701: 'and' in \"and for every\" should probably be removed.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "All in all I believe that this paper provides valuable contribution to the community, as it lays the theoretical foundations for using discrete Bregman divergences in practical learning tasks, which is an intriguing approach that is not often discussed in this context.", "weaknesses": "While the experiments are rudimentary, the paper offers sound theoretical results and demonstrates their applicability to practical learning tasks.\n\nMy main concern is that the paper lacks in terms of clarity, particularly in the experiment section. See details below. Should my concerns be addressed, I would be willing to raise my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)