Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selection
Abstract
Submodular functions and variants, through their ability to characterize diversity and coverage, have emerged as a key tool for data selection and summarization.
Many recent approaches to learn submodular functions suffer from limited expressiveness.
In this work, we propose F LEX S UB N ET , a family of ﬂexible neural models for both monotone and non-monotone submodular functions.
To ﬁt a latent submodular function from (set, value) observations, F LEX S UB N ET applies a concave function on modular functions in a recursive manner.
We do not draw the concave function from a restricted family, but rather learn from data using a highly expressive neural network that implements a differentiable quadrature procedure.
Such an expressive neural model for concave functions may be of independent interest.
Next, we extend this setup to provide a novel characterization of monotone α -submodular functions, a recently introduced notion of approximate submodular functions.
We then use this characterization to design a novel neural model for such functions.
Finally, we consider learning submodular set functions under distant supervision in the form of (perimeter-set, high-value-subset) pairs.
This yields a novel subset selection method based on an order-invariant, yet greedy sampler built around the above neural set functions.
Our experiments on synthetic and real data show that F LEX S UB N ET outperforms several baselines.
Introduction
Owing to their strong characterization of diversity and coverage, submodular functions and their extensions, viz. , weak and approximate submodular functions, have emerged as a powerful machinery in data selection tasks [46, 84, 35, 66, 91, 63, 5, 23, 75].
We propose trainable parameterized families of submodular functions under two supervision regimes.
In the ﬁrst setting, the goal is to estimate the submodular function based on (set, value) pairs, where the function outputs the value of an input set.
This problem is hard in the worst case for poly ( n ) value oracle queries [31].
This has applications in auction design where one may like to learn a player’s valuation function based on her bids [6].
In the second setting, the task is to learn the submodular function under the supervision of (perimeter-set, high-value-subset) pairs, where high-value-subset potentially maximizes the underlying function against all other subsets of perimeter-set.
The trained function is expected to extract high-value subsets from freshly-speciﬁed perimeter sets.
This scenario has applications in itemset prediction in recommendation [78, 79], data summarization [50, 2, 4, 10, 74], etc .
1.1 Our contributions
Driven by the above motivations, we propose (i) a novel family of highly expressive neural models for submodular and α -submodular functions which can be estimated under the supervisions of both (set, value) and (perimeter-set, high-value-subset) pairs; (ii) a novel permutation adversarial training method for differentiable subset selection, which efﬁciently trains submodular and α -submodular functions based on (perimeter-set, high-value-subset) pairs.
We provide more details below.
Neural models for
submodular functions
We
design
F LEX
S UB N ET
, a family
of
ﬂexible neural models for monotone, non-monotone submodular functions and monotone α -submodular functions.
— Monotone submodular functions.
We model a monotone submodular function using a recursive neural model which outputs a concave composed submodular function [27, 76, 52] at every step of recursion.
Speciﬁcally, it ﬁrst computes a linear combination of the submodular function computed in the previous step and a modular function and, then applies a concave function to the result to output the next submodular function.
— Monotone α -submodular function.
Our proposed model for submodular function rests on the principle that a concave composed submodular function is always submodular.
However, to the best of our knowledge, there is no known result for an α -submodular function.
We address this gap by showing that an α -submodular function can be represented by applying a mapping ϕ on a positive modular set function, where ϕ satisﬁes a second-order differential inequality.
Subsequently, we provide a neural model representing the universal approximator of ϕ , which in turn is used for modeling an α -submodular function.
— Non-monotone submodular functions.
By applying a non-monotone concave function on modular function, we extend our model to non-monotone submodular functions.
Several recent models learn subclasses of submodular functions [53, 77, 79].
Bilmes and Bai [7] present a thorough theoretical characterization of the beneﬁts of network depth with ﬁxed concave functions, in a general framework called deep submodular functions (DSF).
DSF leaves open all design choices: the number of layers, their widths, the DAG topology and the choice of concave functions.
All-to-all attention has replaced domain-driven topology design in much of NLP [18].
Set transformers [51] would therefore be a natural alternative to compare against DSF, but need more memory and computation.
Here we explore the following third, somewhat extreme trade-off: we restrict the topology to a single recursive chain, thus providing an effectively plug-and-play model with no topology and minimal hyperparameter choices (mainly the length of the chain).
However, we compensate with a more expressive, trainable concave function that is shared across all nodes of the chain.
Our experiments show that our strategy improves ease of training and predictive accuracy beyond both set transformers and various DSF instantiations with ﬁxed concave functions.
Permutation insensitive differentiable subset selection.
It is common [77, 70, 63] to select a subset from a given dataset by sequentially sampling elements using a softmax distribution obtained from the outputs of a set function on various sets of elements.
At the time of learning set functions based on (perimeter-set, high-value-subset) pairs, this protocol naturally results in order-sensitivity in the training process for learning set functions.
To mitigate this, we propose a novel max-min optimization.
Such a formulation sets forth a game between an adversarial permutation generator and the set function learner — where the former generates the worst-case permutations to induce minimum likelihood of training subsets and the latter keeps maximizing the likelihood function until the estimated parameters become permutation-agnostic.
To this end, we use a Gumbel-Sinkhorn neural network [57, 68, 17, 69] as a neural surrogate of hard permutations, that expedites the underlying training process and allows us to avoid combinatorial search on large permutation spaces.
Experiments.
We ﬁrst experiment with several submodular set functions and synthetically generated examples, which show that F LEX S UB N ET recovers the function more accurately than several baselines.
Later experiments with several real datasets on product recommendation reveal that
F LEX S UB N ET can predict the purchased items more effectively and efﬁciently than several baselines.
Related work
Deep set functions.
Recent years have witnessed a surge of interest in deep learning of set functions.
Zaheer et al.
[86] showed that any set function can be modeled using a symmteric aggregator on the feature vectors associated with the underlying set members.
Lee et al.
[51] proposed a transformer based neural architecture to model set functions.
However, their work do not focus on modeling or learning submodular functions in particular.
Deep set functions enforce permutation invariance by using symmetric aggregators [86, 65, 64], which have several applications, e.g. , character counting [51], set anomaly detection [51], graph embedding design [34, 47, 80, 71], etc .
However, they often suffer from limited expressiveness as shown by Wagstaff et al.
Some work aims to overcome this limitations by sequence encoder followed by learning a permutation invariant network structure [62, 68].
However, none of them learns an underlying submodular model in the context of subset selection.
Learning functions with shape constraints.
Our double-quadrature strategy for concavity is inspired by a series of recent efforts to ﬁt functions with shape constraints to suit various learning tasks.
Wehenkel and Louppe [83] proposed universal monotone neural networks (UMNN) was a signiﬁcant early step in learning univariate monotone functions by numerical integration of a non-negative integrand returned by a ‘universal’ network — this paved the path for universal monotone function modeling.
Gupta et al.
[33] extended to multidimensional shape constraints for supervised learning tasks, for situations were features complemented or dominated others, or a learnt function y =
f ( x ) should be unimodal.
Such constraints could be expressed as linear inequalities, and therefore possible to optimize using projected stochastic gradient descent.
Gupta et al.
[32] widened the scope further to situations where more general constraints had to be enforced on gradients.
In the context of density estimation and variational inference, a popular technique is to transform between simple and complex distributions via invertible and differentiable mappings using normalizing ﬂows [48], where coupling functions can be implemented as monotone networks.
Our work provides a bridge between shape constraints, universal concavity and differentiable subset selection.
Deep submodular functions (DSF).
Early work predominantly modeled a trainable submodular function as a mixture of ﬁxed submodular functions [53, 77].
If training instances do not ﬁt their ‘basis’ of hand-picked submodular functions, limited expressiveness results.
In the quest for ‘universal’ submodular function networks, Bilmes and Bai [7] and Bai et al.
[3] undertook a thorough theoretical inquiry into the effect of network structure on expressiveness.
Speciﬁcally, they modeled submodular functions as an aggregate of concave functions of modular functions, computed in a topological order along a directed acyclic graph (DAG), driven by the fact that a concave function of a monotone submodular function is a submodular function [26, 27, 76].
But DSF provides no practical prescription for picking the concave functions.
Each application of DSF will need an extensive search over these design spaces.
Subset selection.
Subset selection especially under submodular or approximate submodular proﬁt enjoys an efﬁcient greedy maximization routine which admits an approximation guarantee.
Consequently, a wide variety of set function optimization tasks focus on representing the underlying objective as an instance of a submodular function.
At large, subset selection has a myriad of applications in machine learning, e.g., data summarization [4], feature selection [44], inﬂuence maximization in social networks [43, 13, 14, 87], opinion dynamics [11, 12, 14, 89, 49], efﬁcient learning [20, 45], human assisted learning [16, 15, 60], etc.
However, these works do not aim to learn the underlying submodular function from training subsets.
Set prediction.
Our work is also related to set prediction.
Zhang et al.
[90] use a encoder-decoder architecture for set prediction.
Rezatoﬁghi et al.
[67] provide a deep probabilistic model for set prediction.
However, they aim to predict an output set rather than the set function.
Differentiable subset selection.
Existing trainable subset selection methods [77, 50] often adopt a max-margin optimization approach.
However, it requires solving one submodular optimization problem at each epoch, which renders it computationally expensive.
On the other hand, Tschiatschek et al.
[79] provide a probabilistic soft-greedy model which can generate and be trained on a permutation of subset elements.
But then, the trained model becomes sensitive to this speciﬁc permutations.
Tschiatschek et al.
[79] overcome this challenge by presenting several permutations to the learner, which can be inefﬁcient.
One sided smoothness.
We would like to highlight that our characterization for α -submodular function is a special case of one-sided smoothness (OSS) proposed in [29, 28].
However, the signiﬁcance of these characterizations are different between their and our work.
First, they consider γ -meta submodular function which is a different generalisation of submodular functions compared to α -submodular functions.
Second, the OSS characterization they provide is for the multilinear extension of γ -meta submodular function, whereas we provide the characterization of α -submodular functions itself, which allows direct construction of our neural models.
Sample complexity in the context of learning submodular functions.
Goemans et al.
[31] provided an algorithm which outputs a function ˆ f ( S ) that approximates an arbitrary monotone submodular function f ( S ) within a factor O ( √ n log n ) using poly ( n ) queries on f .
Their algorithm considers a powerful active probe setting where f can be queried with arbitrary sets.
In contrast,
Balcan and Harvey [6] consider a more realistic passive setup used in a supervised learning scenario, and designed an algorithm which obtains an approximation of f ( S ) within factor O ( √ n ) .
Design of F LEX S UB N ET
In this section, we ﬁrst present our notations and then propose a family of ﬂexible neural network models for monotone and non-monotone submodular functions and α -submodular functions.
3.1 Notation and preliminary results
We denote V = { 1 , 2 , .., | V |} as the ground set or universal set of elements and S, T ⊆ V as subsets of V .
Each element s ∈ V may be associated with a feature vector z s .
Given a set function
F : 2 V → R , we deﬁne the marginal utility F ( s | S ) :
= F ( S ∪ { s } ) − F ( S ) .
The function F is called monotone if F ( s | S ) ≥ 0 whenever S ⊂ V and s ∈ V \ S ; F is called α -submodular with α >
if F ( s | S ) ≥ αF ( s | T ) whenever S ⊆ T and s ∈ V \ T [35, 88, 21].
As a special case, F is submodular if α = 1 and F is modular if F ( s | S )
=
F ( s | T ) .
Here, F ( · ) is called normalized if
F ( ∅ ) = 0 .
Similarly, a real function f : R → R is normalized if f (0) = 0 .
Unless otherwise stated, we only consider normalized set functions in this paper.
We quote a key result often used for neural modeling for submodular functions [27, 76, 52].
Proposition 1.
Given the set function F : 2 V → R + and a real valued function φ : R → R , (i) the set function φ ( F ( · )) is monotone submodular if F is monotone submodular and φ is an increasing concave function; and, (ii) φ ( F ( · )) is non-monotone submodular if F is positive modular and φ is non-monotone.
3.2 Monotone submodular functions
Overview.
Our model for monotone submodular functions consists of a neural network which cascades the underlying functions in a recursive manner for N steps.
Speciﬁcally, to compute the submodular function F ( n ) ( · ) at step n , it ﬁrst linearly combines the submodular function F ( n − 1) ( · ) computed in the previous step and a trainable positive modular function m ( n ) ( · ) and then, applies a monotone concave activation function φ on it.
Recursive model.
We model the submodular function F θ ( · ) as follows:
[Equation]
where the iterations are indexed by 1 ≤ n ≤ N , { m ( n ) θ ( · ) } is a sequence of positive modular functions, driven by a neural network with parameter θ .
λ ∈ [0 , 1] is a tunable or trained parameter.
We apply a linear layer with positive weights on the each feature vector z s to compute the value of m ( n ) θ ( · ) and then compute m ( n ) θ ( S ) =
(cid:80) s ∈ S m ( n ) θ ( s ) .
Moreover, φ θ is an increasing concave function which, as we shall see later, is modeled using neural networks.
Under these conditions, one can use Proposition 1(i) to easily show that F θ ( S ) is a monotone submodular function (Appendix B).
3.3 Monotone- α -submodular functions
Our characterization for submodular functions in Eq.
(1) are based on Proposition 1(i), which implies that a concave composed submodular function is submodular.
However, to the best of our knowledge, a similar characterization of α -submodular functions is lacking in the literature.
To address this gap, we ﬁrst introduce a novel characterization of monotone α -submodular functions and then use it to design a recursive model for such functions.
Novel characterization of α -submodular function.
In the following, we show how we can characterize an α -submodular function using a differential inequality (proven in Appendix B).
Theorem 2.
Given the function ϕ : R → R + and a modular function m : V → [0 , 1] , the set function F ( S ) = ϕ
( (cid:80) s ∈ S m ( s )) is monotone α -submodular for | S | ≤ k , if ϕ ( x ) is increasing in x and d 2 ϕ ( x ) d x
[Equation]
log
[Equation]
The above theorem also implies that given α = 1 , then F ( S ) =
ϕ ( (cid:80) s ∈ S m ( s )) is monotone submodular if ϕ ( x ) is concave in x , which reduces to a particular case of Proposition 1(i).
Once we design F ( S ) by applying ϕ on a modular function, our next goal is to design more expressive and ﬂexible modeling in a recursive manner similar to Eq.
To this end, we extend Proposition 1 to the case for α -submodular functions.
Proposition 3.
Given a monotone α -submodular function F ( · ) , φ ( F ( S )) is monotone α -submodular, if φ ( · ) is an increasing concave function.
Here, α remains the same for F and φ ( F ( · )) .
Note that, when F ( S ) is submodular, i.e. , α = 1 , the above result reduces to Proposition 1 in the context of concave composed modular functions.
Recursive model.
Similar to Eq.
(1) for submodular functions, our model for α -submodular functions is also driven by an recursive model which maintains an α -submodular function and updates its value recursively for N iterations.
However, in contrast to F LEX S UB N ET , where the underlying submodular function is initialized with a positive modular function, we initialize the corresponding α -submodular function F (0) ( S ) with ϕ θ ( m (0) θ ( S )) , where ϕ θ is a trainable function satisfying the conditions of Theorem 2.
Then we recursively apply a trainable monotone concave function on
F ( n − 1) ( S ) for n ∈ [ N ] to build a ﬂexible model for α -submodular function.
Formally, we have:
[Equation]
with λ ∈ [0 , 1] .
Here, m θ , ϕ θ and φ θ are realized using neural networks parameterized by θ .
Then, using Proposition 3 and Theorem 2, we can show that F θ ( S ) is α -submodular in S (proven in
Proposition 6 in Appendix B).
3.4 Non-monotone submodular functions
In contrast to monotone set functions, non monotone submodular functions can be built by applying concave function on top of only one modular function rather than submodular function (Proposition(i) vs. (ii)).
To this end, we model a non-monotone submodular function F θ ( · ) as follows:
[Equation]
where m θ ( · ) is positive modular function but ψ θ ( · ) can be a non-monotone concave function.
Both m θ and ψ θ are trainable functions realized using neural networks parameterized by θ .
One can use
Proposition 1(ii) to show that F θ ( S ) is a non-monotone submodular function.
3.5 Neural parameterization of m θ , φ θ , ψ θ , ϕ θ
We complete the neural parameterization introduced in Sections 3.2–3.4.
Each model has two types of component functions: (i) the modular set function m θ ; and, (ii) the concave functions φ θ (Eq.
(1) and (2)) and ψ θ (Eq.
and
the non-concave function ϕ θ (Eq.
While modeling m θ is simple and straightforward, designing neural models for the other components, i.e. , φ θ , ψ θ and ϕ θ is nontrivial.
As mentioned before, because we cannot rely on the structural complexity of our ‘network’ (which is a simple linear recursion) or a judiciously pre-selected library of concave functions [7], we need to invest more capacity in the concave function, effectively making it universal.
Monotone submodular function.
Our model for monotone submodular function described in Eq.
(1) consists of two neural components: the sequence of modular functions { m ( n ) θ } and φ θ .
Parameterization of m θ .
We
model the modular function
m
( n ) θ
: 2 V → R + in
Eq
(1) as
m ( n ) θ ( S )
=
(cid:80) s ∈ S θ · z s , where z s is the feature vector for each element s ∈ S and both θ
z s are non-negative
— Parameterization of φ θ .
Recall that φ θ is an increasing concave function.
We model it using the fact that a differentiable function is
concave if its second derivative is negative.
We focus on capturing the second derivative of the underlying function using a complex neural network of arbitrary capacity, providing a negative output.
Hence, we use a positive neural network h θ to model the second derivative
[Equation]
Now, since φ θ is increasing, we have:
[Equation]
Here, φ θ ( · ) is normalized, i.e. , φ θ (0) = 0 , which ensures that { F ( n ) θ } in Eq.
(1) are also normalized.
An offset in Eq.
(5) allows a nonzero initial value of φ θ ( · ) , if required.
Note that monotonicity and concavity of φ θ can be achieved by the restricting positivity of h θ .
Such a constraint can be ensured by setting h θ ( · ) =
ReLU (Λ ( h ) θ ( · )) , where Λ ( h ) θ ( · ) is any complex neural network.
Hence, φ θ ( · ) represents a class of universal approximators of normalized increasing concave functions, if Λ ( h ) θ ( · ) is an universal approximator of continuous functions [36].
For a monotone submodular function of the form of concave composed modular function, we have the following result (Proven in Appendix B).
Proposition 4.
Given an universal set V , a constant (cid:15) > 0 and a submodular function F ( S ) =
φ ( (cid:80) s ∈ S m
( z s ))
where z s ∈ R d , S ⊂ V , 0 ≤ m ( z ) < ∞ for all z ∈ R d .
Then there exists two fully connected neural networks m θ 1 and h θ 2 of width d + 4 and 5 respectively, each with ReLU activation function, such that the following conditions hold:
[Equation]
Monotone α -submodular model.
An α -submodular model described in Eq.
(2) has three trainable components: (i) the sequence of modular functions { m ( n ) θ ( · ) } , (ii) the concave function φ θ ( · ) and (iii)
ϕ θ ( · ) .
For the ﬁrst two components, we reuse the parameterizations used for monotone submodular functions.
In the following, we describe our proposed neural parameterization of ϕ θ .
Parameterization of ϕ θ ( · ) .
From
Theorem 2, we note that ϕ θ ( · ) is increasing and satisﬁes
d 2 ϕ θ ( x ) d x≤ κ ( α ) d ϕ θ ( x ) d x where, κ ( α )
= 1 k log (1 /α ) .
It implies that
[Equation]
Driven by the last inequality, we have
[Equation]
Parameterizing non-monotone submodular model.
As suggested by Eq.
(3), our model for nonmonotone submodular function contains a non-monotone concave function ψ θ and a modular function m θ .
We model m θ using the same parameterization used for the monotone set functions.
We parameterize the ψ θ as follows.
— Parameterization of ψ θ .
Modeling a generic (possibly non-monotone) submodular function requires a general form of concave function ψ θ which is not necessarily increasing.
The trick is to design ψ θ ( · ) in such a way that its second derivative is negative everywhere, whereas its ﬁrst derivative can have any sign.
For x ∈ [0 , x max ] , we have:
[Equation]
where h θ , g θ ≥ 0 .
Moreover, we assume that (cid:82) ∞
h θ ( b ) d b and (cid:82) ∞
g θ ( b ) d b are convergent.
We use x max in the upper limit of the second integral to ensure that ψ θ is normalized, i.e. , ψ θ (0) = 0 .
Next, we compute the ﬁrst derivative of ψ θ ( x ) as:
[Equation]
which can have any sign, since both integrals are positive.
Here, the second derivative of ψ θ becomes
[Equation]
which implies that ψ θ is concave.
Similar to h θ ( · ) , we can model g θ ( · ) =
ReLU (Λ gθ ( · )) .
Such a representation makes ψ θ ( · ) a universal approximator of normalized concave functions.
As suggested by Eq.
(3), ψ θ takes m θ ( S ) as input.
Therefore, in practice, we set x max = max S m θ ( S ) .
3.6 Parameter estimation from (set, value) pairs
In this section, our goal is to learn θ from a set of pairs { ( S i , y i ) | i ∈
[ I ] } , such that y i ≈ F θ ( S i ) .
Hence, our task is to solve the following optimization problem:
[Equation]
In the following, we discuss methods to solve this problem.
Backpropagation through double integral.
Each of our proposed set function models consists of one or more double integrals.
Thus computation of the gradients of the loss function in Eq.
(12) requires gradients of these integrals.
To compute them, we leverage the methods proposed by Wehenkel and Louppe [83], which speciﬁcally provide forward and backward procedures for neural networks involving integration.
Speciﬁcally, we use the Clenshaw-Curtis (CC) quadrature or Trapezoidal Rule for numerical computation of φ θ ( · ) and ψ θ ( · ) .
On the other hand, we compute the gradients ∇ θ φ θ ( · ) and ∇ θ ψ θ ( · ) by leveraging Leibniz integral rule [24].
In practice, we replace the upper limit ( b = ∞ ) of the inner integral in Eqs.
(5), (8) and (9) with b max = x max during training.
Decoupling into independent integrals.
The aforementioned end-to-end training can be challenging in practice, since the gradients are also integrals which can make loss convergence elusive.
Moreover, it is also inefﬁcient, since for each step of CC quadrature of the outer integral, we need to perform numerical integration of the entire inner integral.
To tackle this limitation, we decouple the underlying double integral into two single integrals, parameterized using two neural networks.
— Monotone submodular functions.
During training our monotone submodular function model in
Eq.
(1), we model φ θ in Eq.
(5) as:
[Equation]
In contrast to end-to-end training of θ where φ θ was realized only via neural network of h θ , here we use two decoupled networks, i.e. , φ (cid:48) θ and h β .
Then, we learn θ and β by minimizing the regularized sum of squared error, i.e. , min θ,β (cid:88)
[Equation]
Here, G ( n ) ( S ) = λF ( n − 1) ( S ) + (1 − λ ) m ( n ) θ ( S ) is the input to φ θ in the recursion (1).
Recall that
N is the number of steps in the recursion.
Since φ θ is monotone, we need φ (cid:48) θ ≥ 0 which is ensured by φ (cid:48) θ ( x ) =
ReLU (Λ φ (cid:48)
θ ( x )) .
In principle, we would like to have φ (cid:48) θ ( x ) =
(cid:82) ∞ x
h β ( a ) d a for all x ∈ R .
In practice, we approximate this by penalizing the regularizer values in the domain of interest—the values which are fed as input to φ θ .
While there are potential hazards to such an approximation, in our experiments, the beneﬁts outweighed the risks.
The above parameterization involves only single integrals.
The use of an auxiliary network h β allows more ﬂexibility, leading to improved robustness during training optimization.
The above minimization task achieves approximate concavity of φ via training, whereas backpropagating through double integrals enforces concavity by design.
Appendix C extends this method for α -submodular and non-monotone submodular functions.
Differentiable subset selection
In Section 3.6, we tackled the problem of learning F θ from (set, value) pairs.
In this section, we consider learning F θ from a given set of (perimeter-set, high-value-subset) pair instances.
4.1 Learning F θ from (perimeter-set, high-value-subset)
Let us assume that the training set U consists of { ( V, S ) } pairs, where V is some arbitrary subset of the universe of element, and S ⊆
V is a high-value subset of V .
Given a set function model F θ , our goal is to estimate θ so that, across all possible cardinality constrained subsets in S (cid:48) ⊆
V with | S (cid:48)
| = | S | , F θ ( · ) attains its maximum value at S .
Formally, we wish to estimate θ so that, for all possible
( V, S ) ∈ U , we have:
[Equation]
to
[Equation]
S
[Equation]
4.2 Probabilistic greedy model
The problems of maximizing both monotone submodular and monotone α -submodular functions rely on a greedy heuristic [59].
It sequentially chooses elements maximizing the marginal gain and hence, cannot directly support backpropagation.
Tschiatschek et al.
[79] tackle this challenge with a probabilistic model which greedily samples elements from a softmax distribution with the marginal gains as input.
Having chosen the ﬁrst j elements of high-value-subset from perimeter-set V , it draws the ( j + 1) th element from the remaining candidates in V with probability proportional to the marginal utility of the candidate.
If the elements of S under permutation π are written as π ( S ) j for j ∈ [ | S | ] , then the probability of selecting the elements of the set S in the sequence π ( S ) is given by
[Equation]
Here, F θ is the underlying submodular function, τ is a temperature parameter and S 0 = ∅ .
Bottleneck in estimating θ .
Note that, the above model (16) generates the elements in a sequential manner and is sensitive to π .
Tschiatschek et al.
[79] attempt to remove this sensitivity by maximizing the sum of probability (16) over all possible π :
[Equation]
However, enumerating all such permutations for even a medium-sized subset is expensive both in terms of time and space.
They attempt to tackle this problem by approximating the mode and the mean of Pr( · ) over the permutation space Π | S | .
But, it still require searching over the entire permutation space, which is extremely time consuming.
4.3 Proposed approach
Here, we describe our proposed method of permutation adversarial parameter estimation that avoids enumerating all possible permutations of the subset elements, while ensuring that the learned parameter θ ∗ remains permutation invariant.
Max-min optimization problem.
We ﬁrst set up a max-min game between a permutation generator and the maximum likelihood estimator (MLE) of θ , similar to other applications [68, 62].
Here, for each subset S , the permutation generator produces an adversarial permutation π ∈ Π | S | which induces a low value of the underlying likelihood.
On the other hand, MLE learns θ in the face of these adversarial permutations.
Hence, we have the following optimization problem:
[Equation]
Differentiable surrogate for permutations.
Solving the inner minimization in (18) requires searching for π over Π | S | , which appears to confer no beneﬁt beyond (17).
To sidestep this limitation, we relax the inner optimization problem by using a doubly stochastic matrix P ∈ P | S | as an approximation for the corresponding permutation π .
Suppose Z S =
[ z s ] s ∈ S is the feature matrix where each row corresponds to an element of S .
Then Z π ( S )
≈ P Z S .
Thus, Pr θ ( π ( S ) | V ) can be evaluated by iterating down the rows of P Z S and, therefore, written in the form Pr θ ( P , S ) .
Thus, we get a continuous approximation to (18):
[Equation] [Equation]
so that we can learn θ by continuous optimization.
We generate the soft permutation matrices P using a Gumbel-Sinkhorn network [57].
Given a subset S , it takes a seed matrix B S as input and generates P S in a recursive manner:
[Equation]
[Equation]
Here, t is a temperature parameter; and, D c and D r provide column-wise and row-wise normalization.
Thanks to these two operations, P k tends to a doubly stochastic matrix for sufﬁciently large k .
Denoting P ∞ =
lim k →∞ P k , one can show [57] that
[Equation]
where the temperature t controls the ‘hardness’ of the “soft permutation” P .
As t → 0 , P tends to be a hard permutation matrix [9].
The above procedure demands different seeds B S across different pairs of ( V, S ) ∈ U , which makes the training computationally expensive in terms of both time and space.
Therefore, we model B S by feeding the feature matrix Z S into an additional neural network G ω which is shared across different ( V, S ) pairs.
Then the optimization (19) reduces to max θ min ω (cid:80) ( V,S ) ∈U log Pr θ ( ω, S ) , with ω taking the place of P .
[Table]
Experiments
In this section, we ﬁrst evaluate F LEX S UB N ET using a variety of synthesized set functions and show that it is able to ﬁt them under the supervision of ( set, value ) pairs more accurately than the state-of-the-art methods.
Next, we use datasets gathered from Amazon baby registry [30] to show that F LEX S UB N ET can learn to select data from (perimeter-set, high-value-subset) pairs more effectively than several baselines.
Our code is in https://tinyurl.com/ﬂexsubnet.
5.1 Training by (set, value) pairs
Dataset generation.
We generate | V | =10 4 samples, where we draw the feature vector z s for each sample s ∈ V uniformly at random, i.e. , z s ∈ Unif [0 , 1] d with d =10 .
Then, we generate subsets S of different sizes by randomly gathering elements from the set V .
Finally, we compute the values of F ( S ) for different submodular functions F .
Speciﬁcally, we consider seven planted set functions: (i) Log :
F ( S )
= log( (cid:80) s ∈ S 1 (cid:62) z s ) , (ii) LogDet : F ( S ) =
log det ( I + (cid:80) s
∈ S z s z (cid:62) s ) [73, 8], (iii) Facility location (FL) : F
[Equation]
=
[Equation]
) [58, 25, 19, 61], (iv) Monotone graph cut (Gcut σ =0 . 1 ) :
In general, Gcut σ is computed using F ( S ) :
= (cid:80) u ∈ V,v ∈ S z (cid:62) u
z v − σ (cid:80) u,v ∈ S z (cid:62) u
z v .
It measures the weighted cut across ( S, V \ S ) when the weight of the edge ( u, v ) is computed as z (cid:62)
u z v [37, 72, 39, 40, 41].
Here, σ trades off between diversity and representation.
We set σ = 0 .
Note that Gcut σ is monotone (non-monotone) submodular when σ< 0 .
5 ( σ> 0 . 5) .
(v) Log × Sqrt : F ( S ) =
[log( (cid:80) s ∈ S 1 (cid:62) z s )] · [ (cid:80)
F
S
=
(Gcut
=
8 ) : It is the graph cut in (
) with σ
Among above set functions, (i)– (iv) are monotone submodular functions, (v)–(vi) are monotone α -submodular functions and (vii) is a non-monotone submodular function.
We set the number of steps in the recursions (1), (2) as N = 2 .
Evaluation protocol.
We sample | V | =10000 (set,value) instances as described above and split them into train, dev and test folds of equal size.
We present the train and dev folds to the set function model and measure the performance in terms of RMSE on test fold instances.
In the ﬁrst four datasets, we used our monotone submodular model described in Eq.
(1); in case of Log × Sqrt and
Log ×
LogDet, we used our α -submodular model described in Eq.
(2); and, for Gcut
σ =0 .
8 , we used our non-monotone submodular model in Eq.
For α -submodular model, we tuned α using cross validation.
Appendix D provides hyperparameter tuning details for all methods.
Baselines.
We compare F LEX S UB N ET against four state-of-the-art models, viz.
, Set-transformer [51],
Deep-set [86], deep submodular function (DSF)
[7] and mixture submodular function (SubMix) [77].
In principle, set transformer shows high expressivity due to its ability to effectively incorporate the interaction between elements.
No other method including F LEX S UB N ET is able to incorporate such interaction.
Thus, given sufﬁcient network depth and width, Set-transformer should show higher accuracy than any other method.
However, Set-transformer consumes signiﬁcantly higher GPU memory even with a small number of parameters.
Therefore, to have a fair comparison with the rest non-interaction methods, we kept the parameters of Set-transformer to be low enough so that it consumes same amount of GPU memory, as with other non-interaction based methods.
Results.
We compare the performance of F LEX S UB N ET against the above four baselines in terms of RMSE on the test fold.
Table 1 summarizes the results.
We make the following observations.
(1) F LEX S UB N ET outperforms the baselines by a substantial margin across all datasets.
(2) Even with reduced number of parameters, Set-transformer outperforms all the other baselines.
Set-transformer allows explicit interactions between set elements via all-to-all attention layers.
As a result, it outperforms all other baselines even without explicitly using the knowledge of submodularity in its network architecture.
Note that, like Deep-set, F LEX S UB N ET does not directly model any interaction between the elements.
However, it explicitly models submodularity or α -submodularity in the network architecture, which helps it outperform the baselines.
We observe improvement of the performance of Set-transformer, if we allow more number of parameters (and higher GPU memory).
[Table]
(3) While, in principle, DSF function family contains that of F LEX S UB N ET , standard multilayer instantations of DSF with ﬁxed concave functions cannot learn well from (set,value) training.
5.2 Training by (perimeter-set, high-value-subset)
Datasets.
We use the Amazon baby registry dataset [30] which contains 17 product categories.
Among them, we only consider those categories where | V | > 50 , where V is the total number of items in the universal set.
These categories are: (i) Gear, (ii) Bath, (iii) Health, (iv) Diaper, (v) Toys, (vi) Bedding, (vii)
Feeding, (viii) Apparel and (ix) Media.
They are also summarized in Appendix F.
Evaluation protocol.
Each dataset contains a universal set V and a set of subsets S = { S } .
Each item s is characterized by a short textual description such as “bath: Skip Hop Moby Bathtub Elbow
Rest, Blue : Bathtub Side Bumpers : Baby”.
From this text, we compute z s using BERT [18].
We split S into equal-sized training ( S train ), dev ( S dev ) and test ( S test ) folds.
We disclose the training and dev sets to the submodular function models, which are trained using our proposed permutation adversarial approach (Section 4).
Then the trained model is used to sample item sequence S (cid:48) with preﬁx S (cid:48)≤ K = { s 1 , . . .
, s K } .
Here, s i is the item selected at the i th step of the greedy algorithm.
We assess the quality of the sampled sequence using two metrics.
Mean Jaccard coefﬁcient (MJC).
Given a set T in the test set, we ﬁrst measure the overlap between T and the ﬁrst | T | elements of S (cid:48) using Jaccard coefﬁcient, i.e. , JC ( T ) =
| S
(cid:48)≤| T | ∩ T | / | S (cid:48)≤| T | ∪ T | and then average over all subsets in the test set, i.e. , T ∈ S test to compute mean Jaccard coefﬁcient [38].
Mean NDCG@10.
The greedy sampler outputs item sequence S (cid:48) .
For each T ∈ S test , we compute the
NDCG given by the order of ﬁrst 10 elements of S (cid:48) , where i ∈ S (cid:48) is assigned a gold relevance label 1 , if i ∈ T and 0 , otherwise.
Finally, we average over all test subsets to compute Mean NDCG@10.
Our method vs. baselines.
We compare the subset selection ability of F LEX S UB N ET against several baselines which include two trainable submodular models: (i) Deep submodular function (DSF) and (ii) Mixture of submodular functions (SubMix) described in Section 5.1; two popular non-trainable submodular functions which include (iii) Facility location (FL) [58, 25], (iv) Determinantal point process (DPP) [8]; and, a non-submodular function (v) Disparity Min (DisMin) which is often used in data summarization [10].
Here, we use the monotone submodular model of F LEX S UB N ET .
Appendix F contains more details about the baselines.
We did not consider general purpose set functions, e.g. , Set-transformer, Deep-set, etc., because they cannot be maximized using greedy-like algorithms and therefore, we cannot apply our proposed method in Section 4.
Results.
Table 2 provides a comparative analysis across all candidate set functions, which shows that: (i) F LEX S UB N ET outperforms all the baselines across all the datasets; (ii) DSF is the second best performer across all datasets; and, (iii) the performance of non-trainable set functions is poor, as they are not trained to mimic the set selection process.
Conclusion
We introduced F LEX S UB N ET : a family of submodular functions, represented by neural networks that implement quadrature-based numerical integration, and supports end-to-end backpropagation through these integration operators.
We designed a permutation adversarial subset selection method, which ensures that the estimated parameters are independent of greedy item selection order.
On both synthetic and real datasets, F LEX S UB N ET improves upon recent competitive formulations.
Our work opens up several avenues of future work.
One can extend our work for γ -weakly submodular functions [22].
Another extension of our work is to leverage other connections to convexity, e.g. , the
Lovasz extension [54, 1] similar to Karalias et al.
Checklist
authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Appendix A in supplemental material (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section(b)
Did you include complete proofs of all theoretical results?
[Yes] See Appendix B in supplemental material.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Code is included as part of supplemental material.
Further details are in Appendix D and F in supplemental material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 5, Appendix D and F in supplemental material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Table 1 and Appendix G in supplemental material.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix F in supplemental material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] See Appendix F in supplemental material.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] Yes, we provided code as URL.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A]
No such information present in the data used in this paper.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]