Title: Monotone and Separable Set Functions: Characterizations and Neural Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: DeepSets

Monotone functions

Data Retrieval

separability
Abstract: Motivated by applications for set containment problems, we consider the following
fundamental problem: can we design set-to-vector functions so that the natural
partial order on sets is preserved, namely $S \subseteq T$ if and only if $F (S) \leq F (T )$.
We call functions satisfying this property Monotone and Separating (MAS) set
functions. We establish lower and upper bounds for the vector dimension necessary
to obtain MAS functions, as a function of the cardinality of the multisets and
the underlying ground set. In the important case of an infinite ground set, we
show that MAS functions do not exist, but provide a model called MASNET
which provably enjoys a relaxed MAS property we name “weakly MAS” and
is stable in the sense of Holder continuity. We also show that MAS functions
can be used to construct universal models that are monotone by construction
and can approximate all monotone set functions. Experimentally, we consider
a variety of set containment tasks. The experiments show the benefit of using
our MASNET model, in comparison with standard set models which do not
incorporate set containment as an inductive bias.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses modelling functions on multisets for applications such as Set Containment, and proposes a definition of Monotone-and-Separable functions on such sets. The authors prove under which conditions such functions can exist, and generalize the definition to the case of models with learnable weights. They propose a class of activation functions called 'hat functions' which enforce separability, and define a model architecture which obeys the MAS conditions. They test this architecture on a number of applications related to set containment and demonstrate slightly improved performance compared to existing methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Paper is clearly written and easy to follow (at least for the first half)\n- Theoretical contributions are very good\n\nWeaknesses:\n- Many editing/formatting issues - definition of weak separability on page 5 is missing, Appendix links are broken, tables on page 8 and 9 are small and hard to read and interfere with text. The paper is generally quite cramped near the end and many later sections are overly short and use frequent abbreviations. The paper should likely be reorganized to allow for more room for these final sections, rather than trying to cram them in in a small amount of space.\n- Experiments are limited in scope and results do not show significant improvement over existing baselines. Some of this is fine for a theoretical work, but the results show such little improvement over Set Transformers/Deepsets that it makes me wonder if the strict MAS condition is even necessary in practice.\n- Architecture of MASNet seems incremental, with only very minor differences from DeepSets.\n\nQUESTIONS:\n1) Is there a reason to expect that existing set models fail to obey the separability condition in enough cases to make a model like this necessary? The incremental improvement over simple set baselines makes me suspect that these methods already at least approximately obey the separability condition most of the time in practice. \n2) Why is the condition of monotonicity necessary? The set containment problem only requires separability does it not?\n3) Do you observe significant difficulties in training networks with the 'hat' activation functions? Intuitively I would expect these activation functions to perform very poorly and have significant issues with gradient flow, particularly in deeper networks. It doesn't seem overly important since you show that ReLU-based networks with multiple layers still satisfy separability, but I would expect those activation functions to be almost entirely a non-starter for less trivial cases.\n4) Am I correct in concluding that existing DeepSet models are monotone if they simply enforce non-negativity in their parameters, and separable as long as they have at least 2 layers? This seems like a very minor/incremental change from existing models.\n5) Are Set Transformers with ReLU activations and >2 layers separable? The analysis was largely focused on Deep Sets, which are a much more limited architecture.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of the existence of mapping f between multisets and vector spaces such that S is contained in T (in the multiset sense) if and only if f(S) \\leq f(T) pointwise. The authors consider what is the dimension needed for the existence of such a mapping, consider neural architectures of implementing these functions and test these in multiple experiments on real and synthetic datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper suggests a new notion  of MAS set functions that leads to interesting mathematical analysis. It also provides interesting experiments. One issue is the lack of compelling motivation for MAS as opposed to monotone functions. The authors need to articulate in more detail the motivation for their new notion. I also find the abundance of notions and results a disadvantage making it harder for a reader to take away the important findings and use this paper in theory and practice. The authors may want to focus first on their most important 2-3 findings as opposed to talking about stability, infinite sets  (less relevant to ML) and new activation functions. \nFurthermore the paper is pepped with inaccuracies (line 376, an inaccurate reference to the appendix and more). My feeling is that this paper is not yet ready for publication in a prime conference such as NeurIPS.\n\nQUESTIONS:\n1) in line 31 15 is cited as relevant to the paper. I do not see why it is relevant. Perhaps the authors can explain?\n\n2) Why introduce a new activation function if embedding can implemented by a simple architecture with two layers?\n\n3) The introduction of parameters in the denominators in (8) could lead to numerical instability while training. Can it be avoided?\n\nLIMITATIONS:\n1) Lack of completing motivation for the notion of MAS.\n2) Numerous new notions (MAS, stability, new activation function and more).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors completely characterize the dimensions needed for a “monotone and separating” embeddings (for sets/multisets) in finite settings and show they are impossible in infinite ones, then introduce a relaxed “weakly” version that admits practical neural implementations. They propose MASNET, a simple sum decomposable architecture, based on Deepsets, with specially chosen activations that provably enforces monotonicity and, with high probability, separability. Finally, experiments on synthetic, text, and point cloud tasks demonstrate that MASNET outperforms existing permutation-invariant models on set containment and monotone function approximation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n  - The paper precisely establishes when exact monotone and separating embeddings exist for sets in the finite settings and shows their non existence in infinite ones.\n\n  - Weakly MAS functions are introduced to strikes a practical balance between theoretical rigor and real world feasibility.\n\n  - MASNET combines simple sum decomposable functions with tailored activations to guarantee monotonicity and separability (where possible), as well as introducing the notion of Hölder stability.\n\n  - MAS embeddings are shown to approximate any monotone set function and gives exponential‐decay bounds on separation failure.\n\n  - MASNET consistently outperforms standard permutation‐invariant models on containment and monotone function tasks.\n\n**Weaknesses**\n  \n  - A large portion of the set literature seem to be ignored in this work (see References).\n\n  - The set sizes in the experiments are smaller than what is normally used in the set literature.\n\n  [1] Fspool: Learning set representations with featurewise sort pooling\n  \n  [2] Scalable set encoding with universal mini-batch consistency and unbiased full set gradient approximation\n\n  [3] Enhancing neural subset selection: Integrating background information into set representations\n\n  [4] Mini-batch consistent slot set encoder for scalable set encoding\n\n  [5] HORSE: Hierarchical Representation for Large-Scale Neural Subset Selection\n\nQUESTIONS:\n- Sum decomposable functions like deepsets have been shown limited expressivity compared to attention based methods like Set Transformer. Given that the proposed architecture is based on deepsets, what would be required to propose an architecture based on the transformer like Set Transformer that is monotone and separable? A discussion is enough.\n  \n- How are MAS functions related to the MBC methods [2,4]? From my understanding, the restriction they impose on embeddings of subsets, together with the aggregation functions they utilize (similar to DeepSets) have some connection with the problem tackled in this work.\n  \n- What happens when the set sizes grow to millions? While from Table 6 it seems performance increases with increasing |S|, what happens when |T| is very large?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper discusses embedding (finite) multisets into vectors that preserves monotonicity -- i.e. the embedding of a larger multiset should be a (coordinate-wise) larger vector and conversely that two multisets–each one not a subset of the other–should be mapped to incomparable vectors. The latter direction is termed \"separability\" in the paper.\n\nThe paper explores the minimal vector dimension of such embeddings. It is shown that the minimal dimension is the size of the ground set (the set from which the elements of the multiset are taken). When restricting the function to work on multisets of bounded-cardinality, the minimal dimension may be smaller, and upper and lower bounds for it are derived. It’s also shown that no such embedding exists when the ground set is infinite.\n\nTo allow building such embeddings over infinite ground sets, the paper proposes 2 relaxations of the separability condition:“weakly seperable”, and “lower Holder seperable”. It proves that the embeddings proposed in the DeepSets paper are in general not weakly separable, but claims that using a different activation function it terms a “hat function” or that using more than one-layer of the element embedding part would make the final embeddings weakly separable and lower Holder separable. \n\nFinally, the paper evaluates the benefit of the proposed embeddings to the task of determining set containment and approximating a monotonic set-to-scalar function. It compares with SetTransformer and DeepSet embeddings as the baselines and shows that the embedding proposed perform better.\n\nSTRENGTHS AND WEAKNESSES:\nI think the inductive bias that the paper proposes for “good” multiset embeddings is useful. Containment is a major and general property of set objects and good embeddings should preserve it. To my knowledge its novel.  I find that the paper provides good theoretical characterization of the embeddings and works around the limitations with reasonable weakening of the requirements. \nThe presentation is clear and motivations are provided throught the paper on the various assumptions being made.\n\nWeaknesses:\n- I’m not an expert on partial-ordered sets or lattices, but I’d imagine that the results of Theorem 1 follow pretty easily from standard elementary results in those fields. It’s nice that the proofs are self-contained but it would be good to mention any standard results this can be derived from if any.\n- I would have liked to see more end-to-end experiments of using MAS models to solve realistic modelling problems such as the examples in “Deep Sets”:  point-cloud classification, Text Concept Set retrieval, Sum of Digits, etc. The paper only provides experiments that predict containment which is a much easier task for these embeddings given their inductive bias.\n\nQUESTIONS:\n- It looks like Prop 8 contradicts Prop 6: As far as I can tell, a hat function can be monotone and non-negative, in which case by Prop 6, there exists an S not-subset of T for which F(S,(A,b)) <= F(T, (A,b)) for all A, b, so F is not weekly MAS. Am I missing something?\n- In Theorem 13, isn’t it True that for all injective functions F:P_{<=k}(V) → R^m there is an M such that F(S) =M (F(S) ).  Also you’re using ‘F’ to denote both the MAS and the desired function F. Can you change the notation for one of them to make the theorem statement clearer?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}