Title: Deep Submodular Peripteral Networks
Venue: NeurIPS 2024
Accepted: yes
Keywords: Submodular Optimization

Learning Set Functions

Experimental Design

Streaming Summarization

Data subset selection

Knowledge Distillation
Abstract: Submodular functions, crucial for various applications, often lack practical learning methods for their acquisition.  Seemingly unrelated, learning a scaling from oracles offering graded pairwise preferences (GPC) is underexplored, despite a rich history in psychometrics. In this paper, we introduce deep submodular peripteral networks (DSPNs), a novel parametric family of submodular functions, and methods for their training using a GPC-based strategy to connect and then tackle both of the above challenges.  We introduce newly devised GPC-style ``peripteral'' loss which leverages numerically graded relationships between pairs of objects (sets in our case). Unlike traditional contrastive learning, or RHLF preference ranking, our method utilizes graded comparisons, extracting more nuanced information than just binary-outcome comparisons, and contrasts sets of any size (not just two). We also define a novel suite of automatic sampling strategies for training, including active-learning inspired submodular feedback.  We demonstrate DSPNs' efficacy in learning submodularity from a costly target submodular function and demonstrate its superiority both for experimental design and online streaming applications.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors identify two open problems in machine learning and provide novel solutions to them. The first problem is that even though the submodular functions show up in numerous applications, learning submodular functions through DNNs remain unpractical. Their proposed solution to this issue is their new architecture called Deep Submodular Peripteral Networks (abbreviated as DSPNs). The other identified problem is the lack of graded pairwise preferences (GPCs) oracles. In its essence, it is the type of oracle where given two sets of elements, a score with a scalar value is returned. The sign of this score denotes which set is preferred over the other set and the absolute value of the score denotes the magnitude of this preference. To reflect the effects of this oracle better, they introduce a new loss function called peripteral loss. They evaluate DSPNs ability to learn a target function (a facility location function in the case of their experiments) against different baseline algorithms. They also demonstrate the effects of using different losses when training DSPNs.\n\nSTRENGTHS:\nConsidering the success of supervised learning from comparisons in domains such as healthcare etc., it is no surprise that learning from graded pairwise comparisons is a significant oracle selection. I can see it being very useful in team selection problems where we want to predict the outcome of a competition. The DSPN structure is original. Majority of the ideas conveyed clearly.\n\nWEAKNESSES:\n- I have some concerns related to the organization of the paper. I am aware the page limitations can be annoying sometimes. Still, I do think that Figure 13 should be included in the main body of the paper. I think it would help readers understand the paper better and the main body of the paper would be more standalone that way. Maybe just a 2+1+1 or 2+1+2 layer version of Fig. 13 as you describe in your experiments.\n\n- The introduction mentions E, M sets but we don't know what they stand for until later. You may consider mentioning them as heterogeneous and homogeneous sets before your main contributions.\n\nMinor editorial comments:\n- Appendix or Figure names are not properly capitalized at some places e.g. Line 810.\n- Some figures refer the reader to the appendix without denoting they are in the appendix.\n- On line 96-97, the adjectives need commas in between.\n- Line 844, multiple \"also\"s\n\nQUESTIONS:\n- For my understanding, can we interpret DSPNs as enhanced versions of DSFs with permutation invariance?\n- Where do weighted matroid rank functions stand in relation to threshold potentials (Stobbe and Krause [95])?\n- Isn't Proposition 8 a bit obsolete? Aren't all set functions characterized by their permutation-invariance hence the submodular set functions as well?\n\nLIMITATIONS:\nYes, the authors discuss the limitations of their work in Appendix J.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
This paper proposes a new framework to learn submodular functions.  Specifically, the paper proposes a new parametric family of submodular functions using neural networks. Then, the paper proposes training such networks with a new loss function that is based on graded pairwise comparisons.

STRENGTHS:
I think the paper addresses an important problem of learning/distilling submodular functions in a scalable way. The paper also conducts thorough experiments comparing their approach with baselines.

The paper is well-written and easy to follow. The paper does a good job discussing related and prior work.

WEAKNESSES:
This paper is outside my area of expertise, and as such I don't think I can offer constructive feedback.

QUESTIONS:
None.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
The paper introduces Deep Submodular Peripteral Networks (DSPNs), a novel parametric family of submodular functions designed to address practical learning methods for submodular functions and graded pairwise comparisons (GPC). To learn DSPNs, the paper also introduces a new GPC-style “peripteral” loss, which leverages numerically graded relationships between pairs of objects, extracting more nuanced information than binary comparisons. Finally, the authors demonstrate DSPNs’ efficacy in learning submodularity from a
costly target submodular function showing superiority both for experimental design and online streaming applications.

STRENGTHS:
- The paper introduced Deep Submodular Peripteral Networks (DSPNs), a new parametric family of submodular functions. Since the computational cost for querying a DSPN is independent of dataset size, their approach can learn practical submodular functions scalably.

- The paper also proposed a new GPC-style “peripteral” loss to successfully learn DSPNs. As introduced, this loss has many applications such as learning a reward model in the context of RLHF.

WEAKNESSES:
- It lacks a theoretical guarantee for the proposed peripteral loss. What happens if the target oracle does not exist?

QUESTIONS:
- Can the authors explain intuitively how the peripteral loss is used to learn a reward model in RLHF?

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-14

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces deep submodular peripteral networks (DSPNs) and a graded pairwise preferences (GPC)-style peripteral loss. It shows that DSPNs are effective in learning submodularity from a target function.\n\nSTRENGTHS:\n- The construction of the submodular function, i.e. DSPNs are interesting as they assure the submodularity.\n- The loss function’s construction using contrastive sets is also interesting and reasonable.\n- The experiments demonstrate that DSPNs combined with the peripteral loss are effective.\n\nWEAKNESSES:\nThe paper’s organization needs improvement. Many important concepts, such as submodular functions, FC functions, and GPC, lack definitions (even informal) when they first appear. Additionally, there is no motivation or detailed application of submodular functions in machine learning, making it difficult for readers to understand at the beginning.\n\nMinor: Page 5, line 209, “then so also must”.\n\nQUESTIONS:\n1. I understand that due to page limits, a large portion of the content is included in the appendix. However, I suggest providing a short sentence with informal definitions or references to the definition locations for important concepts so that readers can quickly grasp their meanings and follow the text more naturally. For example, on line 61, what are graded pairwise preferences? Additionally, where is the formal definition of a submodular function? What are the definitions of a matroid and a matroid rank function?\n2. I recommend adding some motivation and detailed applications about how submodular functions are used in machine learning and why they are important. Explicitly explaining where and how submodular functions can be applied in the introduction would be beneficial, instead of generally stating that they have been used in several areas and studied in several papers.\n3. Could the output of the function be a vector of a few dimensions instead of a scalar, where each dimension represents different aspects of comparison, or can be scores judged by different people? Can you provide some insights on whether this scenario is reasonable and if so, whether your framework can be applied to this setting?\n4. In Section 5, what is the target FL and how is it labeled/created? I suggest state explicitly on the task (e.g., image summarization) in the main text.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}