Improving Set Function Approximation with Quasi-Arithmetic Neural Networks
Authors: Tomas Tokar
Scott Sanner
Venue: ICLR 2026 Poster (2026)
Keywords: representation learning
set function learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We introduce quasi-arithmetic neural networks (QUANNs), which use Neuralized Kolmogorov Mean---newly introduced trainable pooling mechanism---to improve set functions approximation.

Abstract: Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such as DeepSets and PointNet rely on fixed, non-learnable pooling operations (e.g., sum or max) -- a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed as more expressive alternatives. In this work, we advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) -- a novel, trainable framework for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as a learnable aggregation function. We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions and, thanks to their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.

Published as a conference paper at ICLR 2026
IMPROVING SET FUNCTION APPROXIMATION WITH QUASI-ARITHMETES NEURAL NETWORKS
Tomas Tokar
Scott Sanner
University of Toronto
University of Toronto
Wondeur AI
Vector Institute
tomas@wcndeur, ai
ssannerêmie utoronto.cc
ABSTRACT
Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such DeepSets and Point- Net rely on fixed, non-learnable pooling operations (e.g., sum or max) a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed more expressive alternatives. In this work, advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) a novel, trainable frame- work for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as learnable aggregation function. We provide the- oretical analysis showing that, QUANNs are universal approximators for broad class common set-function decompositions and, thanks their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.
INTRODUCTION
Sets represent fundamental abstraction for various data types. examples of such include, set of objects placed in scene (Eslami et al. 2016; Kosiorek et al.) 2018), point clouds in physical space (Chang al., 2015 Wu al., 2015), or set of reinforcement learning agents (Sunehag et al. 2017). Developing neural methods that can effectively learn to approximate set functions is therefore crucial for advancing machine learning applications across diverse domains.
Sets impose inherent ordering, making the design of neural architectures for processing them unique challenge. To properly handle this property, neural architectures must exhibit permutation invariance, i.e., ensure that the output remains unchanged regardless of the order of the input el- ements. To guarantee this, most existing approaches such as DeepSets (Zaheer et al., 2017) and PointNet (Qi et al. 2017a) rely pooling operations (e.g. sum, max) that aggregate embeddings of the individual elements or their combinations into single fixed-size representation, which is in turn processed to produce the final prediction (Murphy et 2019) (cf. Figure 1.
Because the pooling operation is fixed and non- trainable, the approximation burden thus divided between the only two neural components: an en- coder o. which maps each element (or their com- binations) to latent representation, and an estima- tor p. which processes the pooled representation to produce the final prediction. Therefore, the encoder forced to learn embeddings that are tailored not only to the downstream task but also to the pri- oni chosen pooling operation, restricting its ability to learn more general transferable latent repre- sentations (Soelch et al. 2019; Wagstaff et al. 2019 Bueno & Hylton 2021
pool
Figure Generalized framework for set function learning, involving encoder o, esti- mator p and pooling operation (e.g. sum, or max).
Wagstaff et al. 2022

---

Published as a conference paper at ICLR 2026
More recently, learnable aggregation functions have been proposed more expressive alterna- tives (Locatello et al. 2020; Pellegrini et al., 2021; Kimura et al. 2024). To advance this line of research, introduce a trainable modification of the Kolmogorov mean (a.k.a. quasi-arithmetic mean) (Bullen & Bullen 2003), an important generalization of means that encompasses many com- monly used measures of central tendency (e.g. arithmetic, geometric, harmonic mean, etc.). The Kolmogorov mean induces set aggregation via selection of invertible generating finction, which implement using an invertible neural network, resulting in highly expressive learnable pooling operation. The use of trainable Kolmogorov mean enables more principled set-function approxi- mation, leading more structured latent spaces, improved encoder transferability, and consistently better downstream performance.
Main contributions (i) We propose the first machine- modification of the Kolmogorov mean. (ii) We propose quasi-arithmetic neural networks (QUANNs), which adopt the newly intro- duced neuralized Kolmogorov mean to serve as a learnable aggregation function. (iii) We support proposal with theoretical and empirical analysis, demonstrating that QUANNs: (a) approx- imate mean- (in the generalized sense) and, under mild assumptions, also max nax-decomposable set functions arbitrarily well; (b) QUANNS enable learning of more structured embeddings and more transferable encoders than existing models; and (c) overall, they outperform state-of-the-ar (SOTA) set function models.
PRELIMINARIES
Set-structured data Let X' denote the input space, and let D ((X; be dataset where each X; is a finite set of elements: X; = (x Xi,,, Xx,r, and y; is the corresponding label. For simplicity, assume that each X; is homogeneous set, i.e., all elements of X; come from the same domain X: X,j E X. However, our framework can be naturally extended to heterogeneous sets. Additionally, restrict our attention the countable case, where all sets X; are finite and of arbitrary cardinality n; that varies across the inputs.
Set function modeling We seek to learn set function F PP)()) + y where Py(X') denotes the set of all finite subsets of X' and the output space y, which is usually either IR (scalar case, e.g. regression), or Rdd for some fixed output dimensionality dest E A (in the vector-valued prediction case, e.g. classification). Our objective is to approximate the function F using neural network, trained on the dataset D by minimizing loss function L obtained from the predicted outputs F(X.) and the labels y; E y.
Permutation invariance A key requirement for function modeling permulation invariance, i.e, the output of the set function should remain unchanged under any reordering of the elements in the input set. Formally, we require the target function estimate F to satisfy:
f(x) = (x)))) VX C X, VT E S
(1)
where Six denotes the symmetric group of all permutations over the elements of the set X, and (X) is the permutation of X under
3 RELATED WORK
3.1 JANOSSY POOLING
Most current methods for learning set functions can be viewed as specific instances of a broader generalization referred to as Janossy pooling (Murphy al. 2019), which can be formalized as follows:
( = P ( (T()))
(2)
where: (X) indicates all l- é-permutations of a set X; encoder Φ is a neural function that learss latent embedding of the input permutations; pool is a permutation-invatie aggregates the obtained embeddings, and estimator p is another neural function that maps an aggre- gated embeddings to the final output.
pooling operation that
2

---

Published as a conference paper at ICLR 2026
Table 1: Unified comparison of the existing methods of Janossy pooling and QUANNs (ours); Φ. and ei are neural functions; w indicates trainable parameter: P3 (X) indicates all 2-permutations (pairs) of the input set and n is set cardinality.
MODEL
FUNCTIONAL FORM
REF
Unary
DeepSets
p( ((x)))
Zaheer et 2017
PointNet
P (max ((x.))
Q al. 201 7a
Normalized DeepSets
p P11/n Zi=1 ((x)))
Bueno & Hylton
2021
Hölder's ' Power DeepSets
p (1/n Ei
m )(xx))))) )
Kimura ct al.
2024
QUANN-1
(1/n Ei= ((((x))))
(ours)
Binary
Se(Transformer
P (xxxx)) ((x,)))))
Lee et al. 2019
QUANN-2
P ( (2xxx))e (00 x,))
(ours)
Any neural network that can be factorized in the above form, is said to be k-ary Janossy pooling, distinct by its choice of k and the adopted pooling operation (cf. Table Below we discuss several important specific cases of Janossy pooling. It is important to note that alternative generalizations of the neural set functions were proposed (Kim et al. 202T).
Unary pooling networks Some of the earliest permutation cermutation-invariant deep learning architectures for set function modeling were DeepSets (Zaheer et al. 2017) and PointNet (Qi et al.) 7a) Both architectures can be viewed as special cases unary Janossy pooling, where the permutation invariance is secured by using summation and max pooling, respectively.
Binary pooling networks Obviously, the unary pooling networks fail to capture the relationships between elements of the set, which imposes an important limitation. Inspired by the success of transformer architecture in various other application, et al. (2019) proposed to addresses this limitation by introducing a SetTransformer, which takes into account the relationships between two elements in the input set via attention mechanism. However, SetTransformer can be factorized into a functional form in the equation with (Wagstaff et al. 2022). and thus constitutes a binary Janossy pooling.
Janossy pooling with learnable pooling function Clearly, the pooling function is the only non- learnable component in Janossy pooling (Eq. [ This was first highlighted in (Soelch et al. (2019) where the authors proposed using recurrent neural networks with attention learnable alternative. However, the method's overall complexity, further aggravated by rather unclear implementation, limits its applicability most real world datasets. Kimura al. (2024) proposed Hölder's Power DeepSets (HPDS) that uses power mean with learnable exponent as the pooling function. Closely related this work the learnable aggregation function proposed by PPeleegiiii et al. (202T), ), in which four distinct power means are fused through a fixed-form aggregation scheme with additional learnable parameters. However, this method does not admit strict Janossy factorization.
3.2
NON-JANOSSY METHODS
Not all inttuutttioniivariann architectures can be factorized into the functional form described in These nan-Jan approaches can be broadly grouped into two conceptually distinct cate- gories: models that achieve permutation invariance through slot attention mechanisms and models that seek an optimal ordering of the input set.
Eq. []
Slot attention Slot attention models learn set representations by binding a fixed number of learn- able "slots" to input set elements, using attention (Locatello et al., 2020). Each slot acts as a latent variable that competes to explain different aspects of the input through soft assignment and recurrent refinement. The input elements are transformed via linear projections and then aggregated through weighted mean, where the attention coefficients serve as the weights. Multiple modification and extensions were later proposed (Skianis et al. 2020 Wu et al. 2022 Jia et al. 2023] Seitzer et al.
2023 Zhhang et al. 2023)
3

---

Published as a conference paper at ICLR 2026
Permutation optimization and sorting Zhang et al. (2019b) addressed permutation invariance by explicitly computing the optimal permutation of the imput set with respect to learnable cost
function, thereby aligning elements before aggregation. In contrast Zhang et al. (2020) proposed simpler approach, where inputs are independently sorted along each feature dimension and subse- quently fed to the encoder.
4 NEURALIZED KOLMOGOROV MEAN
In this section, we introduce neuralized Kolmogorov means (NKM) novel framework for aggre- gating sets of embeddings via learnable measure of central tendency. In the following section we show how NKMs can be used as learnable pooling operation to improve set function approximation.
Kolmogorov mean The Kolmogorov mean, also known as the quasi arithmeetii mean, is measure of central tendency that generalizes various common means. Given a continuous and invertible func- tion f. the Kolmegorov mean of set numbers {ar] is defined as: M [11(1/n Σ. f(x:)): where the function f' is usually referred to generating function. Since the generating function continuous and invertible, it must be a strictly monotonic function on its domain, which guaran- tees that M is bounded between min(x) (x)) and max(x). Special cases of Kolmogorov mean include, arithmetic (f = az + b), geometric (f(x) = log x), and power means (f(z) = x²)).
Neuralization of Kolmogorov mean We propose neuralized form of the Kolmogorov mean, where the generating function is implemented by an invertible neural network tे:
(x(X) +
(xx))
(3)
This way we obtain learnable measure of central tendency, which we refer to neuralized Kol- mogorov mean (NKM). Despite the broad attention given by the scientific community (Kimura) et al. 2024), to our knowledge, we are the first to propose, not only neuralized, but any machine- learnable modification of the Kolmogorov mean. Conceptualy related, yet complementary, is the use of invertible neural functions for neuralization of semirings proposed by Dos Martires (20TT).
Choice of invertible architecture all our experiments, we used the RevNet (Gomez et al, 2017) architecture to implement the generating function & of the NKM. Alternative choice of the invertible architecture NKM were explored, but did not lead to a qualitative change in the models' performance (cf. Appendix 5
5 QUSSI-ARITMEETIC NEURAL NETWORKS
Intuitively, introducing learnable pooling operation to set function models should enhance the ex- pressiveness of their hypothesis spaces. Therefore, we propose adopting the hereby introduced NKMs (cf. Section 4 as learnable pooling operation, resulting in novel class of models, which we refer as Quasi-Arithmett Neural Networks (QUANNs). Note that chose distinct name to emphasize that NKM represents general framework for learnable aggregation of latent codes, whereas QUANNs are specifically designed for set function learning.
5.1 RATIONALE
Any neural network that approximates set function F(X) using the following factorization is referred to as k-ary QUANN, where the value of & distinguishes degree of interactions models:
F(X) ppp
[P(() (1P
Σ b((()))
(4)
(XPL(X)
where p any arbitrary neural functions, while vē is invertible neural function to serve gen- erative function of the NKM. In more compact form, we can write (MM.((X). where M (X) indicates NKM induced via a using encoder d.
4

---

Published as a conference paper at ICLR 2026
Depending on the choice of k thus propose two specific instances of QUANN models: (i) unary pooling network QUANN-1 encoding individual set elements (X) ) = X); and (ii) binary pooling
network QUANN-2 encoding interactions between the set element pairs via attention mechanism (similar to SedTransfomer)(cf. [
5.2 APPROXIMATION OF THE COMMON DECOMPOSITIONS
Theorem 5.1 (Universal Approximation of QUANNs). Let U denote the of all permutation- invariant set functions F PX(()) y that can be uniformly approximated by Quasi-Arithmetic Neural Networks the formld where p. to and are arbitrary neural networks. Let Uw denote the that are uniformly continuous with set functions F P,(()) +
set of all permulation-inaarin respect to the Wasserstein metric. Then U 2 Uw.
Means and central tendencies QUANNs are fundamentally mean-pooling architectures, and as such, they are able to exactly represent any function that is uniformly continuous with respect to the Wasserstein metric (cf. Theorem 5.I).. This means that any continuous function that depends smoothly on the empirical distribution of the input rather than single extreme point, can be approximated arbitrarily well using QUANNS. These involve various types of means and other central tendencies (e.g. median, quantiles).
This also means that QUANNs, principle, are not able to exactly represent target set functions that are sum- or max-decomposable. However, in the context of function learning, exact repre- sentation is seldom the objective; instead, the central goal is often to approximate a target function with sufficient accuracy (Wagstaff' et al., (2022). To this end, we analyze QUANNs approximation capacity with respect to these decompositions.
Max decomposition Under mild assumptions, QUANNs can approximate max-decomposable set functions with worst-case approximation error that scales favorably with the cardinality of the input set (cf. Proposition H.
Approximation of Max- Max-Decooposable Ser Function). This is achieved by learning a generating function vb that falls into an exponential function family (cf. Table 2 In which case, for finite-size sets, QUANNs attain arbitrarily small approximation error.
Sum decomposition QUANNS exhibit limited capacity in approximating sum-decomposable set functions due to the inherently expansive nature of summation, which poses principal challenge for neural architectures designed to preserve per- mutation invariance via normalization-base pool- ing (Bueno & Hylton 202I). Specifically, we demonstrate that, regardless of the particular func- tion & learned by the model, the approximation error
Table 2: Order of approximation & of mean, sum and max-decomposable set functions F(x) that QUANNs can achieve by learning t that falls under the given function family.
F(X)
E
a o mean a max o b
w + W2
0(1)
exp exp (wx)
O (1//w w lo
b
O(n)
tends grow linearly with the cardinality n of the
input set, thus constraining the model's performance in sum-structured tasks (cf. Proposition H.S Approximation of Sum- Sum--ecomposable Set Function). However, QUANNs can still approximate such functions so that the expected approximation error becomes arbitrarity small by letting the encoder "absorb" the expected value of the inputs car- dinality and thus to appropriately rescale the learned NKM (ef. Proposition H.6 Expected value of the Approximation of nam--Decompoable Set Function).
5.3 THEORETICAL BENEFITS OVER NORMALIZED DEEPSETS
A careful reader may notice formal similarity between the functional form of the normalized DeepSet and that of the proposed QUANN model. Specifically, by introducing substitutions p' po ~ and ø' v é, the functional form of QUANNs can be rewritten in form that functionally equivalent to normalized DeepSets: DI o'xx)). This may suggest that QUANNS and Normalized DeepSets are essentially the same model class. However, in such case, p' and ø' would not be independent functions, but functional compositions connected via component s and its inverse. This distinction confers several important advantages the QUANN architecture.
5

---

Published as a conference paper at ICLR 2026
Increased expressivity under fixed p and Φ First, under any fixed choice of the encoder & and the estimator p in a normalized DeepSet, introducing the additional learnable and invertible trans- formation d strictly increases the expressivity of the model. This expansion of the hypothesis space can be particularly valuable in practical scenarios where one or more components (e.g., the encoder Φ) are pre-trained, such in transfer learning.
Natural regularization via dh Enforcing invertibility on sb acts as a natural form of regularization. We hypothesize that this constraint improves the generalization capability of the model by encour- aging the network to learn v that aligns well with the intrinsic structure or decomposition of the target set function F(X); independent of the specific choices of the encoder or estimator networks.
Dimensional collapse prevention To better illustrate this point, consider the dervivative of the function F(X) with respect of the wss the parameter of the encoder ø. For simplicity, we assume that (X) is factorized as QUANN-1 model. Using the inverse function theorem together with the derivative of Kolmogorov mean (cf. Appendix A).
we obtain:
(F( J (M (M
1
0o(x)
Σ (x((())
(5)
Ows
X
aws
XEX
0x(x)
-Jo(Mo.o))
(6)
Ows
xeX
aM 0x(x)
= -M
do(x) Ows
XEX
where J, denotes the Jacobian of the estimator p. During training, the derivative of Kolmogorov mean O.M.o/Oole scales changes in the latent representations of the elements x proportionally their "leverage" on the representation of the entire input set MM.. Because & is invertible, derivative of Kolmogorov mean is non- non-singula,, ensuring that no latent dimension is collapsed to zero when aggregating gradients across the inputs. Consequently, information from each dimension is preserved during backpropagation, effectively preventing dimensional collapse (Jing et al.. 2022)
Structured latent representations e is learned to be locally monotonic around My.ó. then the derivative MM.. (/((()))) in Equation 7 is positive semidefinite. In this case, Mo.e tends to follow the gradients computed across the inputs and stays within a ball around the latent representations of the input elements (Nielsen 2023). leading more structured latent representations.
EXPERIMENTAL DESIGN
Research questions We hypothesize that the theoretical advantages established in the previous section enable QUANNs to learn more transferable encoders and achieve superior performance compared to fatte-of-the-att methods. Based on this hypothesis, we formulate the following research questions. RQ1: Does the proposed neural function factorization described by Equation 4 improve the approximation of common set function decompositions compared an equivalent model with- out the use of invertible neural networks? RQ2: Does the proposed approach learn a better encoder é. by (a) producing qualitatively improved embeddings compared those trained using baseline methods, (b) reducing the need for fine-tuning pre-trained encoders in downstream set function learning tasks, and (c) enabling the learned encoders to be transferable to non-set tasks? RQ3: Do QUANNs outperform state-of-the of-e--o-thearnn art methods in set function learning tasks across diverse experi- mental conditions?
Baselines To evaluate the performance of our proposed method, we compared it against diverse set of Janossy pooling models. We included three unary pooling models: DeepSets (Zaheer et al.) 2017) and PointNet (Qi et al. (2077a). which are widely regarded foundational methods, and Nor- malized DeepSets (Bueno & Hylton 2021). which serves as a representative method for learning average decomposable functions. As a representative of Janossy pooling methods with a learnable pooling function, we employed the Hölder Power Deep Set (HPDS) model (Kimura et al., 2024). For binary pooling architectures, used the SetTransformer (Lee al. 2019). which conveys

---

Published as a conference paper at ICLR 2026
set function approximation via pairwise intention-baseed interactions between set elements. In ad- dition, we compared QUANNs with three representatives of non-Janossy methods: Slot attention
(SlotAtt) (Locatello et al. 2020 FSPool (Zhang et al. 2020) and LAF (Pellegrini et al. 2021
Datasets To evaluate our method against baseline models, we adopted one synthetic dataset and three publicly available real-world datasets: MNIST (LeCun et al.) 1998), Omniglot (Lake et al, 2015) and ModelNet40 (Wu et al.] 2015). Since MNIST and Omniglot, are not originally designed for set function modeling, similar to previous works (Zaheer et al.] 2017 Lee et al. 2019), we introduced modifications adapt these datasets to the set-based setting. The specifie preprocessing procedures and task definitions are detailed in the Section E.1
7
RESULTS
7.1 SYNTHETIC DATA EXPERIMENTS
We designed a synthetic data experiment using randomly generated point clouds and 10 different vector aggregation functions, which models were supposed to approximate (cf. Section E. We evaluated QUANN-1 against three models: Ablation I, in which the transformation & was replaced with the identity function: (x)) = x. Ablation 2, which also replaced & with the identity function,
but extended the encoder Φ and the estimator o networks such that the total number of trainable parameters and the model depth approximately matched those of QUANN-1; and Ablation 3, which
removed Px()x(X) normalization in the Equation QUANN-I outperformed the ablated models in out of 10 experiments (RQ1).
Table 3: Results of the synthetic data experiments. Performance of the proposed model (QUANN-1) and the three ablated models when approximating 10 different set aggregation functions, as mea- sured by mean squared error (MSE, lower is better). QUANN-1 outperformed ablations in 9 out of 10 experiments (highlighted in bold); with significant improvements in 7 of them.
Ablation
Ablation
Ablation 3
QUANN-1
Mean
Std
Mean
Std
Mean
Geometric median
4.521
0.213
14.955
0.704
14.096
1.228
4.138
0.453
Quadratic mean
1.01
0.122
3.643
0.101
6.667
0.805
0.988
0.111
Median
4.628
0.135
15.236
0.781
14.
1.028
4.104
0.383
Medoid
18.725
1.243
Midpoint
25.914
0.571
38.691
0.915
33.578
0.965
24.236
0.497
VecMaxNorm
94.926
137
4.350
114.042
2.363
Max
20.929
0.817
43.143
L.854
45.038
6.135
13,583
0.565
LugEup
19.242
1,078
41.386
1.463
43.969
7.656
12.557
0.856
Variance
132.388
10.578
105. 19
9.594
589,282
95 361
74.128
15.605
Skewness
0.027
0.027
0.001
0.074
0.032
0.027
0.001
7.2 MNIST-SETS EXPERIMENTS
We expanded the sum-of- sum-of-MMIST--iin MNIST-digits experiment, originally performed in Zaheer et al. (2017). by adding other aggregating functions, including max. mean, mode, etc. The model task is to learn to ap- proximate the MNIST digits aggregation function by learning to estimate the function outputs (cf. Fig- ure 2). The experiments were performed under two experimental setups: using generic encoders, which was followed by the qualitative analysis; and using pre-trained encoders (neural function Φ).
#
3
2
Modil
8
34.........2
CasE(B ()
Figure 2: Experiment with aggregation of MNIST digits. The goal is to approximate the function F by learning to estimate its out- puts.
Using generic encoders The MNIST-set experiments were first conducted using generic encoders, where the models had to learn the MNIST image embeddings from scratch alongside other neural components. According to the obtained results (cf. Table 5). QUANN-1 outperformed all unary baselines across all 5 aggregation tasks. The second proposed model, QUANN-2, achieved overall the best performance across all models evaluated. These experiments further expanded by
7

---

Published as a conference paper at ICLR 2026
adding five measures of central tendency; providing further validation of the benefits of QUANN models (cf. Supplementary Table
Qualitative analysis To assess the models' ability learn meaningful input representations, extracted the image encoders o from the Janossy models trained in the previous experiments and applied them to embed the MNIST test set images. The resulting embeddings were subsequently projected into a two-dimensional space using UMAP (MeInnes et al. 20188. The obtained projec- tions were then plotted reveal the structure of the embeddings (cf. Fi The results show that only the embeddings obtained from QUANNs, and SetTransformer, but not other models, consis- tently produced class-wise separable structures across all tasks, demonstrating their ability to learn more structured representations (RQ2a).
The experiments were then repeated using pre-trained MNIST image encoders (neural function é). Specifically, we first trained a standalone MNIST image classifier composed of a small convolutional neural network followed by an MLP predictor, achieving 98% test set accuracy. The trained encoder was transferred to the set function models, with its weights being fixed. The models were then trained in the same manner as in the previous experiments.
Using pre-trained encoders
Since the encoder is trained to produce class-separated embeddings and not to recover quantitative relations between the images, we expected that the performance of the set function models would generally worsen under this setting. The goal was to assess whether QUANNs would experience a smaller performance drop compared to the baselines. This was confirmed by the results, which show that only QUANNs retain their performance across all tasks, whereas the baselines experience significant performance drop (cf. Figure [ despite similar number of trainable parameters (RQ2b).
7.3 OMNIGLOT-SETY EXPERIMENTS
We modified the experiment proposed in Lee al. 2019) to multi-label binary classification, where the goal was identify which of the 40 Omniglot alphabets were represented within a given set of Omniglot handwritten character images (ef. Fig- ure 3). The experiments were conducted under four different conditions, fixing the minimum cardinality at Mmin = 5 while varying the maximum cardinality of the image sets as 7max = 10, 15, 20 and 25.
R
2
g
Xooid
Hebrwer
Sunders
Cace(y 8)
Figure Omniglot experiment. The task is to identify which alphabets are represented in set of images of
characters.
The performance was assessed using the balanced accuracy score, computed as the macro-average across the alphabets. The obtained results (cf. Table show that QUANNs surpassed all baselines.
Transfer learning We extended the Omniglot periments with a transfer learning analysis. We ex- tracted the encoders & from all models trained in the Omniglot experiment, froze their weights, and trans- ferred them a image classification model com- posed of the encoder followed by linear layer. This model was trained to predict the alphabet which each image belongs.
Table 4:
Omniglot image classification ob- tained from the transfer learning experiment.
ACC
Mean
Std
Model
Unary
DeepSet
0.524
0.042
PointNet
0.256
0.219
NormDeepSet
0.393
0.052
The results show that encoders learned by QUANNs provide better support for the image classification compared to those learned by the baselines (cf. Ta- 4l This suggests that the factorization employed in QUANNs learns more transferrable embeddings that can be used beyond the set-related tasks (RQ2a).
HPDS
0.435
0.023
QUANN-1
0.597
0.014
Binary
Se(Transformer
0.480
0.019
QUANN-2
0.589
0.041
Non-Janossy
FSPool
0.463
0.051
SlotAut
0.484
0.018
LAF
0.156
0.007
7.4 NED EXPERIMENTS
We performed multi-class classification experiments using the ModelNet40 dataset, following a
setup similar to that of Qi et al. (2017a) and Qi al. (207b)). The task was predict the category
8

---

Published as a conference paper at ICLR 2026
of the object represented by point cloud. The experiments were conducted under three different settings: the point clouds were subsampled to n points, where the maximum number points was fixed at Rmma 2048, while the minimum number of points varied as Mmmn 256, 512 and 1024.
QUANNS were outperformed by FSPool and also by PointNet, but not other baselines (cf. Table 5) The high performance of FSPool and PointNet is not surprising, both models employ architectures that are well-suited for detecting extreme values (max or min), a property known to be critical for capturing local and global geometric features of point clouds (Zhang et al. 2019a). Overall, these results corroborate the findings reported in the original experiments (Qi et al. 2017a
7.5 QM9 EXPERIMENTS
Finally, we evaluated our models on two regression tasks derived from the QM9 molecular dataset (Ramakrishnan et al. 2014). Each molecule is represented as set of atoms with associ- ated 3D coordinates, ranging from 3 to 29 atoms per molecule.
the first task, the goal was to predict the HOMO energy level, while in the second task we predicted the LUMO energy level. As shown in Table QUANN-2 achieves the best performance across all baselines on both tasks.
DISCUSSION
Conclusions In this work, we introduced QUANNS, novel approach for set function approximation that leverages neuralized Kol- mogorov mean as a trainable pooling operation. We presented theoretical benefits of this construction and hypothesized that these advantages should promote the learning of more transferable latent representations and improved generalization. The obtained empirical results provide evidence in support of these hypotheses and show that QUANNs significantly outperform their state-of-the-arr alternatives (ef. Figure (RQ3).
DeepSer-
[0:2
0.02
0.0
#Shool 4:7
0.68
65-10.14
0.21
0.5
0.02
0:03
2:47
0:21
663
206
60
0.5
Ieetthe: 2:37 0:25
0.37
855
0.50
221
0.15
0.42
3
OUAM 2 1. wompape Loe L.tmtssso
Aethanstome aaee w7r0 sero
0.
021
Seste
3.
Fanso
Posttee
sendhe arddenttenn
Limitations As the major limitation of QUANNs we consider their limited capacity approximating sum-decomposable set functions (cf. Section 5.2) Therefore, if the target set function is expected be additive or expansive, simple sum may provide bet- ter alternative to NKM. Similarly, when data are lim- ited, NKM may be unnecessarily flexible and prone to overfitting, whereas a fixed function may provide better inductive bias and computational simplicity.
Figure 4: Relative number of outcomes (ef. Table 5) where the model in the given row outperformed the model in the given column. An asterisk and a double asterisk indicate sta- tistical significance (p < 0.05) and very high significance (p- < 0.01), respectively.
Similar to most previous works, our thcoretical and experimental results are restricted to countable sets. Intuitively, extending QUANNs to uncountable sets may be possible by replacing the sum in Eq. 4 with an appropriate form of computational integration. However, we cannot guarantee which of the theoretical benefits of our method would carry over in this setting, and the approximation of set functions over uncountable domains remains an open question.
Applications QUANNs provide a general and expressive framework for learning central tenden- cies over arbitrary collections of representations, making them potentially useful across a variety of domains. For instance, QUANNS could be employed within graph neural networks, replacing conventional message-passing aggregators with learnable generalized mean, which is fundamen- tally different from the approaches of [ooro et al. (2020) and Ong & Veličković (2022), who use learnalle fusion of simple aggregates and learnable pairwise operations, respectively. Similarly, in multi-modal and multi-view learning (Guo et al. 2019). where information from several modalities sources must be fused, QUANNs offer principled way to learn the fusion rule for combining representations from each view. QUANNs could also be applied in federated learning (Guendouzi et al. 2023), where model updates from multiple clients must be aggregated.

---

Published as a conference paper at ICLR 2026
Table 5: Model performance across all experiments, grouped by model class (Unary in the upper ble; Binary and Non-Janossy in the lower). All values report mean performance across experimental replicates with corresponding standard deviations. Bold and underlined values denote the best and second-best performance across all models, respectively.
(a) Unary models
DeepSet
PoiniNet
NormDeepSet
HPPDS
QUANN-1
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Mean
Std
Agg.
MNIST, MSF.
max
0.120
0.012
0,079
0.002
0.128
0.014
0.071
0.009
0.067
0.022
mide
3,903
0.150
5,498
0.270
4.119
0.170
4.011
0.295
2.219
0.163
5.844
variance
1.747
0.226
2.143
0.121
1.927
0.132
1,265
0.414
0.605
0.171
Agg.
MNST Prr trained encoders, MSE
max
0.421
0.199
0.018
0.314
0.068
mean
0.207
0.008
0.317
0.023
0.201
0.005
0.153
0.006
0.021
0.003
mode
6,384
0.170
3.626
0.095
3.300
0.085
sum
15,241
1.129
83.217
2.177
17.281
1,756
18.115
2.764
3.074
0.113
variance
0.086
3,766
0.182
3.950
3.774
0.296
0.625
0.212
Omniglot, Rolonced ACC1
10
0.579
0.009
0.566
0.077
0.544
0.011
0.610
0.017
0.734
0.007
0,523
0.036
0.535
0.015
0,711
0.017
20
0.588
0.007
0,515
0.012
0.527
0.002
0.543
0.011
0.674
0.033
0.592
0.004
0,545
0.077
0.536
0.006
0.537
0.009
0.001
ModelNes40. ACC
256
0.648
0.007
0.788
0.005
0.649
0.006
0.658
0.009
0.676
0.003
512
0.649
0,788
0.002
0.520
0.667
0.007
1024
0.645
0.012
0.791
0.006
0.534
0.007
0.663
0.013
0.686
0.013
Task
Q44, MSE
homo
179.423
10.715
153.419
7.299
695
6.495
212.237
7.899
160.345
12.676
lumo
399.119
13.734
401.247
3.917
458.711
10.189
674.320
20.098
374.646
9.096
(b) Binary & Non-Janossy models
Binary
Non-Janossy
SelThansfrrmer
QUANN-2
FSPaol
SlotAnt
L.A
Mean
Std
Mean
Std
Mean
Std
Mcan
Std
Mean
Std
Age.
MNIST, MSE
max
0.099
0.009
0.069
0.014
0.095
0.018
0.085
0.025
0.571
0.092
0.031
0.002
0.023
0.003
0.075
0.007
0.029
0.005
0.050
0.012
mode
3.085
0.367
1.579
0.408
2.847
0.479
3.255
0.124
4.219
0.370
4.090
0.601
2,861
0,251
5.202
0.441
66.324
5.957
20.920
3.681
variance
2.284
0.298
0.605
0.043
1.317
0.201
1.956
0.120
1.383
0.197
Agg.
MNIST Pre-trained encoders, MSE
max
0.079
0.016
0,042
0.008
0.230
0.038
0.077
0.013
0.653
0.045
0.049
mode
2,837
0.185
0.756
0.169
3.475
0.381
2,556
0.178
2,851
0.911
9,520
0.509
variance
2,322
0.351
0.616
0,134
2.583
0.241
1.825
0.026
1,775
0.321
Tleass
Omniglot, Bolanced ACC1
0.525
0.074
15
0.603
0.003
0.711
0.015
0.579
0.079
0.618
0.005
0.528
0.003
0.6016
0.060
25
0.606
0.006
0.635
0.069
0.640
0.012
0.616
0.005
0.545
0.010
Phein
ModelNer40, ACC1
256
0.676
0.006
0.773
0.011
0.796
0.011
0.726
0.003
0.660
0.020
512
0.679
0.009
0.762
0.011
0.791
0.013
0.739
0.014
0.671
0.028
1024
0.678
0.009
0.782
0.005
0.794
0.012
0.727
0.012
0.688
0.008
Tisk
Q449, MSE
homo
195.998
2.392
104.821
10.472
176.240
7.769
164.640
19.453
297.712
84.158
lumo
461.948
11.837
258.039
20.471
480.726
32.999
386.630
11.782
657.083
77.112
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean Government (MSTT) (No. RS-2024-00457882, National AI Research Lab Project).
REFERENCES
Christian Bueno and Alan Hylton. On the representation power of set pooling networks. Advances in Neural Information Processing Systems, 34:17170-17188 2021.
Peter S Bullen and PS Bullen. Quasi-arithmetic means. Handbook of means and their inequalities, pp. 266-320 2003.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint /120v:/51..03012. 2015.
Gabriele Corso, Luca Cavalleri, Dominique Beaini, Pietro Liò, and Petar Veličković. Principal neighbourhood aggregation for graph nets. Advances in neural information processing systems, 33:13260-1322 2020.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components esti- mation. arXiv preprint arXiv:1410.8516, 2014.
Laurent Dinh, Jascha Dick-Dic,s,,in, and Samy Bengio. Density estimation using real nvp. arXiv preprint vrXiv:1605.088000 2016.
Pedro Zuidberg Dos Martires. Neural semirings. In NeSy. International Conference Neurosym- bolic Learning and Reasoning, 2021.
SM Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, David Szepesvari, Geoffrey E Hinton, et al. Attend, infer, repeat: Fast scene understanding with generative models. Advances in neural information processing systems, 29, 2016.
Aidan N Gomez, Mengye Ren, Raquel Urtasun, and Roger Grosse The reversible residual net- work: Backpropagation without storing activations. Advances in neural information processing systems, 30, 2017.
Badra Souhila Guendouzi, Samir Ouchani, Hiba EL Assaad, and Madeleine EL Zaher. A systematic review of federated learning: Challenges, aggregation methods, and development tools. Journal of Network and Computer Applications, 220:103714 2023.
Wenzhong Guo, Jianwen Wang, and Shiping Wang. Deep multimodal representation learning: A 2019.
survey. leee Access,
Baoxiong Jia, Yu Liu, and Siyuan Huang. Improving object-centric learning with query optimiza- tion. In International Conference on Learning Representations, 2023.
Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse in contrastive supe-supe learning. In 10th International Conference on Learning Representa- tions, ICLR 2022, 2022.
Jinwoo Kim, Saeyoon Oh, and Seunghoon Hong. Transformers generalize deepsets and can be extended to graphs & hypergraphs. Advances in Neural Information Processing Systems, 34: 28016-28028, 2021.
Masanari Kimura, Ryotaro Shimizu, Yuki Hirakawa, Ryosuke Goto, and Yuki Saito. permutation-invariant neural networks. arXiv preprint arXiv:2403. 174.00 2024.
Adam Kosiorek, Hyunjik Kim, Yee Whye Teh, and Ingmar Posner. Sequential attend, infer, repeat: Generative modelling of moving objects. Advances in Neural Information Processing Systems, 31, 2018.
11

---

Published as a conference paper at ICLR 2026
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-lever concept learning through probabilistic program induction. Science, 350(6266):1332- 1338, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-23 1998.
Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-basee permutation-invant invariant neural networks. In Interna- fional conference machine learning, pp. 3744-3753. PMLR, 2019.
Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot atten- tion. Advances in neural information processing systems, 53:11525-11538 2020.
Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv: /802.03426, 2018.
Murphy, B Srinivasan, V Rao, and B Riberio. Janossy pooling: Learning deep permutation- invariant functions for variable-size inputs. In International Conference on Learning Representa- tions (ICLR 2019), 2019.
Frank Nielsen. Beyond scalar quasi-arithmetic means: Quasi-arithmetic averages and quasi- arithmetic mixtures in information geometry. arXiv preprint arXIv:2301.10980. 2023.
Euan Ong and Petar Veličković. Learnable commutative monoids for graph neural networks. In Learning on Graphs Conference, pp. 43-1. PMLR, 2022.
Giovanni Pellegrini, Alessandro Tibo, Paolo Frasconi, Andrea Passerini, Manfred Jaeger, et al. Learning aggregation functions. In IJCAI, pp. 2892-2898. International Joint Conferences on Artificial Intelligence Organization, 2021.
Charles R Qi. Hao Su, Kaichun Mo, and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652 660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical fea- ture learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017b.
Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1(1):1-7,
Nimrod Segol and Yaron Lipman. universal equivariant set networks. 8th International Conference on Learning Representations, ICLR 2020, 2020.
M. Seitzer, Horn, A Zadaianchuk, Zietlow, Xiao, C Simon-Gabriel, T He, Z. Zhang, B Schölkopf, Thomas Brox, et al. Bridging the gap to real-world object-centric learning. In International Conference on Learning Representations, 2023.
Konstantinos Skianis, Giannis Nikolentzos, Stratis Limnios, and Michalis Vazirgiannis. Rep the set: Neural networks for learning set representations. In International conference on artificial intelligence and statistics, pp. 1410-1420. PMLR, 2020.
Maximilian Soelch, Adnan Akhundov, Patrick van der Smagt, and Justin Bayer. On deep set learn- ing and the choice of aggregations. In Artificial Neural Networks and Machine Searning-ICAAN 2019: Theoretical Neural Computation: 28th International Conference on Artificial Neural Net- works, Munich, Germany, September 17-19, 2019, Proceedings, Part I 28, pp. 444 457. Springer, 2019.
Peter Sunchag, Guy Lever, Audrumas Gruslys, Wojeiech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Mare Lanctot, Nicolas Sonnerat, Joel Leibo, Karl Tuyls, et al. ceooppscconooitioon networks for cooperative multi-agent learning. arXiv preprint arXiv: 1706.02266 2017.
12

---

Published as a conference paper at ICLR 2026
Edward Wagstaff, Fabian Fuchs, Martin Engelcke, Ingmar Posner, and Michael A Osborne. On the limitations of representing functions on sets. In International Conference on Machine Learning, pp. 6487-6494. PMLR, 2019.
Edward Wagstaff, Fabian B Fuchs, Martin Engelcke, Michael A Osborne, and Ingmar Posner. Uni- versal approximation of functions on sets. Journal of Machine Learning Research, 23(151):1-56,
2022.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912-1920, 2015.
Ziyi Wu, Nikita Dvomik, Klaus Greff, Thomas Kipf, and Animesh Garg. Slotformer: Unsupervised visual dynamies simulation with object-centric models. In International Conference Learning Representations, 2022.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and processing systems, 30, 2017.
Alexander Smola. Deep sets. Advances in neural processnioonrrccesinnng
Binbin Zhang. Shikun Huang, Wen Shen, and Zhihua Wei. Explaining the pointnet: What has been learned inside the pointnet? In CVPR Workshops, pp. 71-74, 2019a.
Lily Zhang, Veronica Tozzo, John Higgins, and Rajesh Ranganath. Set norm and equivariant skip connections: Putting the deep in deep sets. In International Conference on Machine Learning, pp. 26559-26574. PMLR, 2022.
Yan Zhang, Jonathon Hare, and Adam Prügel-Bennett, Learning representations of sets through optimized permutations. In International Conference on Learning Representations, 2019b.
Yan Zhang, Jonathon Hare, and Adam Prüigl--Benett.. Fspool: Learning set representations with featurewise sort pooling. In International Conference on Learning Representations, 2020.
Yan Zhang, David W Zhang, Simon Lacoste-Julien, Gertjan Burghouts, and Cees GM Snoek. Unlocking slot attention by changing optimal transport costs. In International Conference Machine Learning, pp. 41931 1991-11151. PMLR, 2023.
13

---

Published as a conference paper at ICLR 2026
A DERIVATIVE OF THE KOLMOGOROV MEAN
We analyze the derivative of the Kolmogorov mean with respect to single input element 2y. Given an invertible and differentiable transformation y) the Kolmogorov mean of a X (x....... x,
is defined as:
Mx(X)
f(xi))
(8)
Let z, fxx;) and define the aggregated representation 2 - Z, Then:
y = My f
Our goal is to compute the sensitivity of y with respect to a single input xy. Applying the multivariate chain rule:
(M,(X)
0z)(2z)
Oz
0x,
8z
0x,
The second term is straightforward:
Oz 1 oz
1
- ))((,),
(10)
0xj n 0x; 71
where Jf(x,)
af(x;) 8xy
is the Jacobian of f at Xj.
For the first term, use the inverse function theorem: the Jacobian of the inverse map is the inverse of the Jacobian of the forward map:
0 05- 2
J, (z)=
where y = My))).
(11)
8z
Combining these results, we obtain:
(M, 0x
Jz(1))
(12)
Key properties Since is invertible by the definition of Kolmogorov mean, its Jacobian Jy is non- singular. Therefore, the resulting derivative ( (X) is also guaranteed to be nin-singular. Moreover, if ʃ is also monotonic, the resulting derivative is positive (in the scalar case) has a positive-definite symmetric part (in the multivariate case). The resulting derivative of the Kol- mogorov mean thus inherits least two important structural properties:
Dimension preservation (from non-singularity) - NKM maps any infinitesimal input per- turbation 8x, to output perturbation dy so that 5y is non-zero along all non-zero dimen- sions of 6x,. The NKM thus acts as a smooth aggregator that maps small input neighbor- hoods to small latent neighborhoods so that no dimension is flattened to zero.
Local alignment (from cesitive-defiiitnnsss NKM maps any infinitesimal input pertur- bation dx, to an output perturbation ōy that satisfies ōy ox; > 0 ensuring that small input perturbations produce output changes locally aligned with the input shift.
B CHOICE OF N-NETWORK ARCHITECTURE
In all our experiments, we used the RevNet (Gomez et al. 2017) architecture to implement the generating function z of the NKM. RevNet is composed of sequence of invertible blocks, where each block operates on a partitioned input vector (21, x22 and is defined as:
1 = σ + f(2))
(13)
2/2 = x2 + g((())
14

---

Published as a conference paper at ICLR 2026
where f and are arbitrary neural networks (we used ReLU-activated MLPs), and the transformation easily invertible by sequentially subtracting these updates in reverse. We selected RevNet for its simplicity and efficiency, making it intuitive choice for learning sb.
To further investigate the relationship between the performance QUANN and the architectural design of the neural network implementing the generating function v/ repeated the synthetic data experiments (ef. Section (7.1 using only QUANN models, under different architectures imple- menting the generating function vb.
Use of RealNVP coupling The RevNet blocks used to implement the generating function v were replaced with conceptually similar but more expressive RealNVP coupling blocks (Dinh al. 2014 2016). This modification preserved the invertibility constraint while increasing the representational capacity vb. The results (cf. Figure 5) did not reveal clear consistent pattern indicating improve- ment or worsening of the model performance, indicating that while architectural enhancements may yield marginal gains some tasks, overall effectiveness QUANNs does not critically depend on the specific choice of invertible block, provided it is sufficiently expressive and scalable.
Varying RevNet capacity We systematically varied the number of RevNet blocks and the number of hidden layers per block. The resulting performance metrics, summarized in Figure 0 indicate that the impact of these architectural choices is highly dask-dependent. That is, no single configuration consistently outperforms others across all aggregation functions. These findings suggest that, in practice, it is not possible to determine the optimal design of & a priori. Instead, users are advised empirically explore different configurations in order identify the most effective architecture for a given task.
w Architecture
Geometric median
Quadratic mean
Median
Medoid
Midpoint
MSE
RevNett
RealNVP
RewNet
RealNVP
RevNet
RealNVVP
RevNet
ReaiNVP
RevNet
RealNVP
VecMaxNorm
Max
LogSumExp
Variance
Skewness
90.0
87.5
RealNVP
RevNet
RealNVP
RevNet
RealNVP
RealNVP
Figure 5: QUANN performance in synthetic data experiment in dependence the choice of the di-network architecture (RevNet vs RealNVP coupling). The results show that there is no clear consistent pattern indicating improvement or worsening of the model performance due altered network architecture.
C SCALABILITY OF NEURALIZED KOLMOGOROV MEANS
The NKM computation scales linearly with the number of elements in the input set: O(n). involving forward passes through the generating function vb, followed by single pass through its inverse 2/-1. The scaling behavior with respect to the input dimensionality, d, depends on the choice of neural architecture used to implement v. When
realized as a RevNet, the memory and com- putational costs scale linearly with d: O(d). Overall, these properties make NKM a practical and scalable choice for processing even large and high-dimensionall input sets.
D POSSIBLE EXTENSIONS & MODIFICATIONS OF QUANNS
In experiments, purposefully refrained from using regularization techniques adopting more complex architectures. to maintain a clear epistemological focus on the core properties of the proposed models. Nevertheless, identify least three promising avenues for extension and
15

---

Published as a conference paper at ICLR 2026
Hidden Layers (y)
[32, 32]
[32]
[64]
Geometric median
Quadratic mean
Median
Medoid
Midpoint
25
4.0
4.0
17.0
3.5
0.3
16.5
24
blocks
VecMaxNorm
Max
LogSumExp
Variance
Skewness
15
14
MSE
14
75
0.027
68
13
50
0.026
10
04
Num. Dlocks
Num. blocks
Num. blocks
Num. blocks
Num. blocks
Figure 6: QUANN performance in synthetic data experiment in dependence on the choice of the vi-network architecture (number RevNet blocks and hidden layers per block). The results show that there is not universal relationship between the choice of the architecture and the associated performance.
modification of our work: (i) the adoption of regularization techniques, particularly set-specific nor- malization layers such as SetNorm (Zhang et al. 2022); (ii) hierarchical extensions of QUANNs, analogous how PointNet++ (Qi et 2017b) extends the original PointNet (Qi et al. 201 and (iii) the introduction of "multi-head" learnable pooling, where multiple Neuralized Kolmogorov Means (with independently trained generating functions) are applied parallel and then concate- nated, may substantially boost expressivity of the models.
D.1 DOMMUTAATION-QQIVARRIII QUANNS
Our proposed framework focuses on learning permutation-innarian functions. A common strategy to extend permutation innarriannn models to with global, sum-pooled features and then apply an clement-wise function combine local and global information (e.g., (Segol & Lipman 2020 Analogously, permutation-equivaiii QUANNs layer can obtained by augmenting elements after applying the inverse transformation:
ones is to augment each element
f(X) = P Xi. 10-
Σ
Mo(xi))
(14)
P&(
TEPA(X)
To evaluate the performance of permutation- equivariant layers, we conducted the following experiment. We implemented the permutation- equivariant variant of DeepSets as described in Za- heer al. (2017) (Section 3.1, Equivariant Model, and Appendix C). We then assessed this model under different pooling operations using the ModelNet40 dataset, strictly adhering to the preprocessing proto- col outlined in Zaheer al. (2017) (Section H), in- cluding www--maaannnnttrriaaaa
Table 6: ModelNet40 classification obtained by the permutation--qquiariiie models.
ACC1
n = 100
n 1000
Model
Mcan
Std
Mcan
Std
DeepSct
0.798
0.021
0.844
0.017
QUANN-1
0.786
0.015
0.835
0.022
normalization and fixed set cardinality (n 100 and n = 1000). The best performance we obtained closely matched the results originally reported in Zaheer al. (2017).
QUANN-2
0.846
0.012
0.882
0.018
We then implemented an analogous model in which the equivariant DeepSets layers were replaced
with QUANN- -UseN equivariant layers defined in Equation 14 While the equivariant QUANN-1 achieved performance comparable to that of equivariant DeepSets, the equivariant QUANN-2 model substantially outperformed both, shown in Table
16

---

Published as a conference paper at ICLR 2026
E EXPERIMENTS DETAILS
E. DATASETS
Synthetic dataset Each point cloud X = (x:) consisted of n E N vectors randomly sam- pled from x E 10. Number of vectors n was sampled uniformly from interval [2, 1024]. The obtained vectors were subsequently transformed using affine transformations of the form az + b, where a E (0, 1) is a scalar and (-10, 10) '6 is a translation vector sampled independently for each cloud. To assign labels, we applied a selected aggregation function F(X) IR + R com- puted across the set of transformed points. We experimented with total of ten distinct aggregation functions, encompassing range of statistical descriptors (cf. Table 8). The resulting dataset was partitioned into training, validation, and test subsets consisting of 20 x 103, 2 x 103, and 3 x 10 clouds respectively.
MNIST-sets dataset MNIST is a widely used benchmark dataset consisting of grayscale 28 x 28 pixels images of handwritten digits (0-9) (LeCun et al. 1998). We constructed a set dataset derived from the MNIST images and their corresponding digit labels. Each instance in our dataset was created by a set of randomly selected MNIST images of varying sizes, where the set label was obtained by applying an aggregation function the set of associated image labels. The number of elements in each set, n, was sampled uniformly between Pmsnn = 2 and maa = 16. We employed ten different aggregation functions, including common measures of central tendency (e.g., mean, median), extremal values (max), summation, variance, and others. The task was to estimate the set labels. For each aggregating function we generated a new datasets, including training, validation and testing subsets; consisting of of 20 x 10 2 x 103, and 3 x 10 instances respectively.
Omniglot-sets dataset Omniglot is a collection of handwritten characters drawn from approxi- mately 50 different alphabets, containing over 1,600 unique character classes, originally proposed for evaluation of few-shot learning (Lake et al.) 2015). Following similar procedure to the MNIST-set dataset, we constructed variable-sized sets from the Omniglot dataset by randomly sam- pling images from the Omniglot collection. Each set assigned binary label vector indicating which of the 50 alphabets were represented in the set, based on the presence or absence of their characters. This framing naturally leads to a binary multi-label classification task, where the ob- jective is to predict which alphabets appear in a given set. We sampled between Ummin and Ymmoz images, where the minimum set size was fixed at Rmmnn 5 was maximum was varying across Umaz E {10, 15, 20, 25} to explore the effect of increasing the upper bound on the set size on the models' performance. For each value of Amaxx we generated a new dataset comprising training, validation, and test subsets, consisting of of 20 x 103, 2 x 10², and 3 x 10 instances, respectively.
ModelNet40 dataset ModelNet40 is a 3D object recognition dataset containing CAD models from 40 object categories, commonly used in point cloud classification tasks (Wu et al. 2015). Each ob- ject is represented as point cloud consisting of a varying number of 3D coordinates. We uniformly subsampled each object to obtain a fixed number of points n E {256, 512, 1024). The resulting point clouds were then normalized to fit within unit sphere centered at the origin, ensuring geo- metric consistency across all instances. The classification task involved predicting the correct object category from the point cloud representation. For each value of n. we generated new dataset com- prising training, validation, and test subsets, consisting of 20 x 10 2 x 10 and 3 x 103 objects, respectively.
QM9 dataset QM9 consists of computed geometrie, energetic, electronic, and thermodynamic properties for 134,000 stable small organic molecules made up of C, H, O. N, and F (Ramakrishnan et af. 2014). These properties were obtained through computational simulation and, although they are close to experimentally measured values, the dataset cannot be strictly considered woal-world. Each molecule consists of a variable number of atoms with associated 3D coordinates. The number of atoms ranges from 3 to 29, so the dataset to be viewed naturally as a set-valued input of varying cardinality. We conducted two independent prediction tasks: estimating the HOMO and LUMO energy levels, whose values stability. For each task, we trained and evaluated the models using independent training, validation, and test splits, comprising 80%, 10%, and 10% of the data, respectively.
converted to milli-electron volts (m for improved numerical
17

---

Published as a conference paper at ICLR 2026
E.2 MODELS IMPLEMENTATION
All models, including our proposed QUANNS, its ablated variants, and the baseline methods, were implemented in PyTorch. The baselines were implemented based the methodological descriptions provided in their respective papers and, when available, their official code repositories. Every effort was made reproduce their original implementation faithfully and accurately.
To ensure a fair comparison across models, we standardized the encoder & and estimator p archi- tectures used in all experiments (cf. Table [Tp. This consistency maintained across all models, thereby isolating the impact of the aggregation mechanism as the primary variable under investiga- tion.
Furthermore, to ensure comparable learning capacity between QUANN-1/-2 and the unary or bi- nary aggregation-based baselines, we intentionally reduced the capacity of QUANNS' encoder and estimator networks. Specifically, one hidden layer was removed from each of these components in QUANNs. This adjustment led all models having approximately the same number of learnable parameters, which mitigates any performance differences that could otherwise be attributed to model size (cf. Table 12
E.3 TRAINING & TESTING
In all experiments, each model was trained using only the training portion of the dataset. During training. we explored a range of learning rates in order to identify optimal training configurations. Following training, model selection Specifically, for each model and learning rate configuration, we measured the validation loss and selected the trained instance that achieved the lowest validation method was selected, final evaluation was conducted on the test portion of the dataset. The resulting test loss or, where applicable, another performance metric was recorded. Information about hyper- paramters and loss functions selection are summarized in the Table I0
performed based on performance on the validation set.
Once the best model for each
E.4 STATISTICAL EVALUATION
Performance evaluation For each real-world dataset experiment, we performed independent experimental replicates. For synthetic data experiments, we conducted 10 replicates. All reported values represent the mean and standard deviation computed across these experimental replicates. To assess whether one model significantly outperformed another, adopted commonly used informal heuristic; a model was considered perform significantly better if the difference in mean performance exceeded the sum of the corresponding standard deviations.
Win-loss matrices The models were additionally compared in pair-wise fashion and the results were conveyed as win-loss matrix (Figure 4.. summarizing the total number of outcomes where the model in the given row outperformed the model in the given column (comparing mean values only, irrespectively of the standard deviation), normalized by the total number outcomes the models participated in (cf. Table
Binomial test To assess the statistical significance of the win proportions in the win-loss matrix, one-tailed binomial test was conducted for each entry. The test designed to evaluate whether the observed proportion of wins for given model is greater than what would be expected by chance. The null hypothesis thus says that observed proportion of the wins r is not greater than 0.5 the expected proportion under random chance:
Ho : 0.5
H1 : 0.5
The entries with the p-ralue < 0.05 were considered significant (*), and those with p-value < 0.01 considered highly significant (**).
18

---

Published as a conference paper at ICLR 2026
E.5 COMPUTATIONAL RESOURCES
The experiments were performed across 4 NVIDIA RTX A6000 GPUs, and 28 AMD EPYC-Rome Processors, Ubuntu 20.04.6 LTS (Focal Fossa). The project code is available https://
/tLtabb..com//mmmatttrrrr/aass http://uuasiAAiirtttt.e.aa ireer Neural -aerall-etworss
F SUPPLEMENTARY TABLES
Table 7: Performance of individual models in the additional MNIST-set aggregation experiments. All values represent the Mean Squared Error (MSE; lower is better), averaged across multiple perimental replicates, with standard deviations reported. Values highlighted by bold indicate best performance, while those highlighted by underline show the second best performance.
Unary
[liory
Non Jannesy
DeepSet
NormDeepSett
QUANN-I
StTTransfermer
QUANN-2
LagMecenFop
0.016
0,000
0.005
median
0.042
0.680
0.028
0.382
0.346
0.137
midringe
0.015
0,002
0.024
Table 8: Summary of the point cloud aggregation functions used in the synthetic data experiments (cf. Section 7.1). Each function maps sets of vectors IR to single representative vector (R* Rd). Each function captures different statistical geometric properties of the input point set. Type
Function Name
Expression
Central lendencies
Marginal median
(mediann (r.,
Geometric median
min
||x: z|/i
Medoid
arg min Cx
xx:
x
f=1
1/2
Quadratic
Extremes
Midpoint
(x: + xx)),
where (i.5) arg max ||x: x,112 in
VecMaxNorm
arg max ||>:||2
2,Ex (max(xx (x:.;)))) 5))
Row max
LogSumExp
log
exp(x::)
mml
Moments
Variance
[x, - x)²
Skewness
1=1
19

---

Published as a conference paper at ICLR 2026
Table 9: Summary of the labels aggregation functions used in the aggregatonnof--MiSTimmmgg experiment. Each function operates on set of scalars {I1, (2.,,,,.
Function Name
Formula
Mean
Median
median(21 .2,... mode( 22.
Mode
1/m
Geometric Mean
T
73
Harmonic Mean
Log-Mean-Exp
log
min; Tc + max, 26
Midrange
2
Variance
Σ (xq x)?
where -n 24
1=1
Maximum
max 25
Sum
1=1
Table 10: The hyper-parameteer and loss function selection as used in our experiments.
Dataset
Learning Rate
Batch Size
Epochs
Latent Dim
Loss
Performance Metric
Synthetic data
1.0 x 10-4
32
20
16
MSE
MSE
5.0
1,0 X 10
5.0 10
MNIST-set
1,0 10 4
128
MSE
MSE
5,0 x 10
1.0 10
Omniglot-set
1.0 10
128
BCE
Ballanced ACC
5.0 x 10
1.0 10
ModeINet40
1.0 10 10
32
50
128
MSE
ACC
5.0 x 10
256
1.0 X 10
20

---

Published as a conference paper at ICLR 2026
Table 11: Summary of encoder v and predictor architectures used for individual modalities across the given datasets. Note, dim indicates latent space dimension.
Datasct
Encoder Ø
Estimator
Synthetic
Input: R
Input: R Rd
FC 128 ReLU
FC 128 ReLU
dém
MNIST-set
Input: R- RXx25x25
Input: dd FC 256 ReLU
Conv kernel 3, stride pad ReLU
Conv 32, kernel 3, stride 1, pad + ReLU
FC 256 ReLU
Conv 64, kernel 3 x 3, stride 1, pad + ReLU
FC
MaxPool2d
PC dém
Omniglot-set
Input: R
Input: Rd
Conv 16, kernel 3 x 3, stride 1, pad ReLU Conv 32, kernel 3, stride pad + ReLU
FC 256 ReLU FC 256 ReLU
Conv 64, kernel 3, stride I, pad + ReLU
MaxPool2d
FC dém
ModelNet40
Input: R
Input: Ramm
FC 256 + ReLU
FC 256 ReLU
FC 256 ReLU
FC 256 ReLU
FC dim
PC 40
Table Number of trainable parameters per model across all experiments.
Synthetic
MNIST
MNIST-
Omniglot
ModelNet40
QM9
Ablation
8480
Ablation 2
15776
Ablation 3
11696
DeepSet
196225
99073
208818
208808
200065
NormDeepSet
196225
99073
208818
208808
200065
HPDS
196226
99074
208819
208809
200066
QUANN-1
11696
196737
99585
209330
209320
134785
SerTransformer
263553
166401
269874
271144
267393
QUANN-2
246657
149505
252978
320552
316801
PSPool
196245
99093
208838
208036
134293
SlotAtt
281217
184065
287538
288808
285057
LAF
197761
100609
210354
210344
201601
21

---

Published as a conference paper at ICLR 2026
G SUPPLEMENTARY FIGURES
mean DeepSet
mode DeepSett
variance DeepSet
max DeepSet
sum DeepSet
10
30
10
0-
10
10
20
-10
10
10
10
mean NormDeepSet
mode| NorüDeepSet
variance NormDeepSet
max NormDespSet
sum NormDeepSet
30
10
10
10
10
mean |
PointNet
mode PointNet
variance Pointhiet
max | PointNet
sum PointNet
UN002
10
30
5.0
0-
0.0
10
10
20
20
mean HPDS
mode HPDS
variance | HHDSS
max HPDS
sum HPDS
10
UNAP2
5.0
2.5
10
10
-5
10
10
mean QUANN-1
made QUANN-1
variance QUANN-1
max | QUANN-1
sum QUANN-1
15
UMAP2
10
0-
-10
10
-10
10
20
20
moan SetTransformer
mode Settransffmmrr
variance SetTransformer
max SetTransformer
sum SetTransformr
15
UMAP2
20
0-
O-
10
15
10
10
-5
10
mcan QUANN 2
mode QUANN
variance: QUANN 2
max QUANN 2
sum QUANN
UMAP
10
10
UMAP
UMAP1
LIAP
UMAP
UMAP
Figure 7: UMAP projections of the MNIST test set embeddings obtained from the encoder networks (ő) trained under different Janossy methods to approximate various aggregation functions. Each point corresponds to image, colored by its digit class label. Notably, only the encoders trained via Se(Transformer. QUANN-1 (unary pooling), and QUANN-2 (binary pooling) produce embeddings that exhibit clear class-wise separation over all tasks, highlighting their superior ability to organize inputs a semantically meaningful latent space.
22

---

Published as a conference paper at ICLR 2026
MNIST experiments: generic-s-pre--triinn encoders;
Max
Meon
Mode
Sum
Varfance
De- Te-tineed arester
Deeptet
HPD5
NormDeepiit
PorntNet
SetTanstormer
PSPool
QUANN-3 QUANK-2
Figure 8: Set function models trained with generic vs. a fixed, pre-trained MNIST image encoder. Subplots correspond to different aggregating functions, and show the model performance (MSE, lower is better), with error bars indicating standard deviations across experimental replicates. Only QUANN-1 and -2 retain, below, the identity line), whereas baselines experience a significant performance drop (points above the identity lines); despite comparable number trainable parameters (cf. Table 12)
improve, their performance despite fixed encoder (points close to, or
23

---

Published as a conference paper at ICLR 2026
H PROOFS
Corollary H.1 (Permutation invariance of Quasi-ane arithmetii Neural Networks), Let X be a finite set of elements as described Section 3. and let F(X) be a set function approximation as described in Eq. 4 Then, F is permutation invariant with respect to the elements of the input set X.
Proof. For any permutation x of the indices {l,.... n1), we have:
(0- 0
E ((llEns))) (0 0-1 1 Σ (olees) F(X),
F(X')
since summation is invariant under permutation. Thus, F is permutation invariant.
Corollary H.2 (Kolmogorov Mean with Linear Generator is Equal to Arithmetic Mean). Let (x) be Kolmogorov mean as defined in Equation 3 If 1b IR + R is linear function of the form (x) = wix + wa for constants wi + and W2 R, then My is equal to the arithmetic mean.
Proof. Let (x)) = w + u2, with wi * O, so that db is invertible with inverse (1) = (y w2)/u/ W1. The Kolmogorov mean is then computed as:
n
(I mn)= = --
((as) = =
W 2 + w2)
Yi
Simplifying the expression inside the inverse:
1al + W2) = wi
+ u2
1=1
and apply -b:
W1 (1 Σ
+ W12
W2
w1
+ u22 =
24.
11
W1
Therefore,
1 Σ
(x)
84.
11
Proof of therorem 5.7 Let's decompose U as the following union:
u = U4 Ul
where U denotes set of functions that can be uniformly approximated by QUANNs of the form equation with = I, while p. Φ are arbitrary neural networks.
By the representation result of Bueno & Hylton (2021), every permutation-invaaiie set function F (X) + y that is uniformly continuous with respect to the Wasserstein metric can be uniform approximated by function of the form
F(X) P
Σ xEX
d(a))
X
where p and & are arbitrary chosen neural networks.
This representation corresponds the QUANN architecture specialized to I (i.e. the quasi- arithmetic mean reduces to a sum or mean when z the identity) and so U = Ulw
From the above obtain that 24 Us=1 UU Uw Ul and therefore:
u 2 Uw.
24

---

Published as a conference paper at ICLR 2026
Proposition H.3 (Approximation of Mean-Decone Decomposable Set Function). Let {I15.... In be a finite set as defined in Section 2 and consider a mean mean- decooposabll set function F(x) of the form:
F(X)
Σ b(zz))
where:
bb:- + Rd is a continuous function;
Rd gd is Lipschitz continuous function with Lipschitz constant La. for some E. E.
Let FX) denote the approximation of F(X) produced by the model described in Section which constructs:
continuous function Φ = b,
generator to approximaling any linear map + Wi + and
continuous function a.
Then, for any 0, if approximates linear function sufficiently well and both & and p approx- imate and a, respectively, within corresponding tolerances, the approximation error is bounded
|F(X)- - -(x)] E.
Proof. Let μb == 1
n Ei= b(x;) and uøo == の Σ (o(x;))) The approximation error
be expressed
[F(x) - F(X)I [a(p)) p(u
By the triangle inequality and the Lipschitz continuity of a, along with the universal approximation property (i.e., [la(z) - p(=)l - cp for any 2), we have:
[F(XX) F(X) = |(a(u)) p(Ho,))ll
(15)
|a(H)) a (Hv.,))) [|a(Hv.)) p(
(16)
La 11 Mll + Epo
(17)
where L, is the Lipschitz constant of
We now focus on bounding I|/6 - Mobool. 1l By definition:
llpb Hwooll
1
b(x)))))
b(x.)
R Φ-
H p(o(a)))
n
7
From Corollary H.2 if ab approximates linear function (xx) z wix + uy such that [|b())
(wir + w2)ll S 6x. then in the limit as c - O, the Kolmogorov mean reduces to the arithmetic mean:
lim Pe +0
((x)).
n
Using this and the universal approximation property of o. i... [[b(z) x(x)
S cd. we can bound:
lim ||/b - Mv.o -0
((x))
((.))
(18)
n
1=1
(b(x;) - ((zx))
(19)
Σ [bb()z) - ((x))1
(20)
c
(21)
25

---

Published as a conference paper at ICLR 2026
Substituting into the earlier inequality:
lim
S Loeo +
Finally, since both e and eo can be made arbitrarily small, we can write:
lim [IF(X) F(X)I se
for arbitrarily small e 0.
Proposition H.4 (Approximation of Max- dax-Decooposable Set Function). Let X = fx11-... Fnd be fifintte set as defined in Section 2 and consider a mean-decomee dreeo-decomppsable set function F(x) of the form:
F(X)
max (naxb((z) ZE
where:
b: X R.d. a continuous function;
Rdd Rdd is Lipschitz continuous function with Lipschitz. constant Lax for some
d, d' e N.
Let (X) denote the approximation F(X) produced by the model described in Section which constructs:
continuous function ó = b,
generator & approximating any exponential function x exp(wxr) s.t. 0. and
a continuous function R2 a.
Then, for any 0, if 1/ approximates an exponential family function sufficiently well and both and p approximate and a, respectively, within corresponding tolerances, the approximation error bounded as:
[[F(X) - f(x)) < La log (n) +c.
Proof. Let M, maxzex ((x) and poo. 10 EE ((((x)))) The approximation error can be expressed as:
[F(X) - F(X)I = (a(M)) p(uo,))
(22)
By the triangle inequality and the Lipschitz continuity a, along with the universal approximation property of (i.e., [|a(z) p(=)1l Ep for any z), we have:
[[((X) F(X)| (la(M)) p(po,)))
(23)
< [la(M)) a(Hp.o)l + [la()))o) + plug,e))l
(24)
< La Mo - uwooll +
(25)
where Lg is the Lipschitz constant of a.
We now focus on the bounding ||,r Me.ell ||. By definition:
M,, Ho
M₁ 1/
Σ (((()))
(26)
ZEX
26

---

Published as a conference paper at ICLR 2026
When d approximates exponential function (x) z exp (wx) such that (10()))cxp) (wz))l S cΦ. then in the limit as Es - 0, the above norm can be computed as:
1 w
lim M Mveel 6-
M
log
exp(wo((x))
(27)
log
exp(wo(x))) -Mb
(28)
i=i
exp(w<(x))) 1
log
log (exp (wc
(29)
imi n exp(wo(x) wM))
log
(30)
(31)
We will now establish the lower bound on the sum inside the logarithm. We will leverage the approximation boundary [1ő(x) b(x) S cs: and that b(x) My for least one x E X.
exp(wΦ(x) - wM) = exp(w(0(x) Mb)
(32)
12
n exp(u(b(z) Mb EG))
(33)
exp(w(b(xx) Mb Eo))
(34)
#E h(m)
2 exp w
(35)
n
Given this bound establish the upper bound the limit of the above norm:
exp(wő(x) - wM) 2 exp( (ww))
lim 60
Il - n0.0l S
log (n) + Ex (36)
Substituting into the earlier inequality:
lim [I - F(X)Il s La( log (n) + es) + -0
(37)
w
Finally, since both 6o and e, can be made arbitrarily small,
write:
1 lim [|x(X)) - F(X)I < La log (n) + -0
(38)
w
for arbitrarily small & O.
Proposition H.5 (Approximation of Sum-Decone Deoo-eecoppsaale Set Function). Let X (x finite set as defined in Section 2 and consider sum-decooposable set function F(x) of the form:
., Ind } be
F(X) = a Σ b(xs)
where:
. b X Rdd is continuous function bounded by the interval (Bo, B1);
27

---

Published as a conference paper at ICLR 2026
R + Re is Lipschitz continuous function with Lipschitz constant Lu, for some E. E.
Let F(X) denote the approximation of F(X) produced by the model described in Section
which
constructs:
a continuous function @ Re wbb w > O,
continuous function a.
Then, for any c > O, i approximates any continuous invertible function and both and p approx- imate functions wh and respectively, within corresponding tolerances, the approximation error is bounded as:
[[((X) - F(X)II S Ta (nB1 -Boo) +
Proof. Let So = Ei= b(x;) and Pv.v ( 1 The approximation error can be expressed
F(X) - F(X)I = |a(S)) p(HN00)
(39)
Same in the proofs of Propositions
H.3 and H.4
[[F(X) F(X)
|la(S)) p(
(40)
$ |lo(Sb) - a(He, + ( + p(p
(41)
La Ms c
(42)
Since wb(z) Es 6(x) wb(x) + Eg and Bo b(x) < B1 then leveraging that Kolmogoov
mean is neither larger than the largest input, nor smaller than the smallest input (cf. Section 4 we obtain: wBo- Co < Pv.s < w B1 + co-
Given this bound we establish the upper bound on the above norm:
SS Mvrøll = [s -- ( Σ M((+)))
(43)
DEX
I |l BB call Substituting into the earlier inequality:
(44)
[X(x) F(X)I S L.n(B wB)) cb) c
(45)
Finally, since both es and en are arbitrarily small, we can write:
[I((x) - ((X)) S La(nBI - wB) +
(46)
Proposition H.6 (Expected value the Approximation of Sum- cum--ecomposable Set Function), (Ex-
tending the Proposition H.S) Assuming cardinality n of the set X follows independent distribution p(n), with ñ being its expected value, then, for any c > O, if 1/s approximates any invertible con- tinuous function and both & and p approximate functions nBB,/ Bo b and a, respectively, within corresponding tolerances, the expected value of the approximation error is bounded as:
[[I F(X)I] Se
(47)
Proof. From the previous proposition we obtain that:
( < Ln(B wBo)
(48)
The expected value of the approximation error
[[II((X) -Fx)(x LaB -BBp) +
(49)
28

---

Published as a conference paper at ICLR 2026
When the scale w, learned by phi, approximates the n.11/ / Bo: RB1/BB b, such that ||0 nB1 Bu bll < ew then in the limit Ew + O, the above expected value bounded by:
lim E (XI F(X)II I Laew +
(50)
Finally, since both and 6ur are arbitrarily small, we can write:
lim [x) F(X)I] s c0
(51)
for arbitrarily small 0.
29


Reviews:
[{"review_id": "qNlws6PVIC", "reviewer": "Reviewer_oz2P", "summary": "This paper introduces quasi-arithmetic neural networks (QUANNs) for learning set functions. By parameterizing the Kolmogorov mean function (also called the quasi-arithmetic mean, and hence the name of the proposed architecture), QUANNs can be seen as an extension of the Normalized DeepSets architecture, where the standard mean function in Normalized DeepSets is replaced with a learnable Kolmogorov mean. More generally, QUANNs further allow aggregation over subsets of set elements akin to the Janossy pooling. The authors provide explanation for the theoretical benefits QUANNs, as well as hypotheses for the practical advantage of QUANNs. The author verify their hypotheses by carrying out a set well designed experiments using both synthetic and real-world data. Empirically, QUANNs are shown to outperform state-of-the-art set neural network baselines.", "questions": "For the synthetic data experiment, Ablation 2 should in theory be strictly better than Ablation 1. Why do the results in Table 3 show the opposite? Was it because of bad training, bad tuning/regulations, or else?\n\nAlso for the synthetic data experiment, many of the functions involve computing some average in one way or another. This aligns with the normalization used in QUANN. What happens if you want to learn functions that are not averages, but sums (e.g. vector norms)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea to parameterize the Kolmogorov mean function and replace the standard mean pooling in existing set neural network architectures is intuitive and interesting. Empirically, the resulting QUANN architecture outperforms all baselines. It is good to see that a simple and principled modification leads to effective performance gain.\n\nThe experiments are well designed to empirically verify the authors' hypotheses on the practical advantages of QUANNs.", "weaknesses": "Writing can be improved. \n- Section 5.2 has a lot of references to the theorems in the appendix. It is difficult to follow the discussion in this section without having to read the theorems in the appendix. I think it would be helpful to at least state some informal and short versions of the theorems here. \n- Please use citet{…} and citep{…} appropriately, e.g., use citep{…} for references that appear in lines 28-32. There are many misuses of citet{…} or cite{…} throughout the entire paper, please fix these. In Line 116, should define S_k(X) here, not P_k(X). Be consistent in the use of notations in Equation (2) and (4).\n\n\nI am not sure how practical the new architecture is. The authors should discuss the practical relevancy of set representation learning in the current state of machine learning. Can't LLMs/vLLMs solve the tasks considered in the experiments?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v9oxvhHPo0", "reviewer": "Reviewer_71eq", "summary": "The paper presents a novel approach for learning the pooling operator in set function approximation. By leveraging a neuralized version of the Kolmogorov mean, it manages to approximate various measures of central tendency.", "questions": "Can you clarify the main differences wrt alternative approaches for learning pooling operators?\n\nCan you discuss more clearly the implications of the countability assumption?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and clear. The choice of Kolmogorov mean and its neutralized version is well-motivated. There is a sound discussion of the theoretical advantage of the proposed solution over alternatives, complemented with promising experimental results.", "weaknesses": "There were few works that explicitly addressed the problem of learning pooling operators in the past:\n\n- Euan Ong, Petar Veličković, Learnable Commutative Monoids for Graph Neural Networks, LOG 2022.\n- P. Zuidberg Dos Martires, Neural Semirings, NeSy 2021.\n- G Pellegrini, A Tibo, P Frasconi, A Passerini, M Jaeger, Learning aggregation functions, IJCAI 2021.\n\nWhile none of them directly suggests using the Kolmogorov mean, they all attempt to go beyond predefined aggregators, and at least one (neural semiring, arguably a not very popular paper) explicitly mentions the use of invertible neural networks. Positioning the contribution with respect to these works would better clarify its novelty.\n\nCountability assumption. This is not really a weakness (most existing results, starting from deep sets, assume countable sets), but given its relevance (sets of real vectors are uncountable) it would be worth discussing the assumption in the limitations.\n\nMinor: \n- please check your citation reference fornatting and use citep when appropriate. \n- Pk(X) -> Sk(X) [or viceversa] in the description of Eq. 2", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gCeJeDsDkS", "reviewer": "Reviewer_ZtCS", "summary": "This paper presents Universal Set Transformer (UST), which combines a set transformer with mini-batch consistency (MBC), resulting in a methodology that is both expressive and has the ability to handle large sets. Set and elements can be represented, performance is often better than baselines, and explainability is possible by means of element-wise attention scores.", "questions": "What is the difference between the multiset attention as presented in this paper, compared to ref [1] (see above)? If this distinction is significant, and can be motivated in technical/math terms, then I am willing to increase my score, because it is a good paper but this remaining point needs clarification.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The formulation of element-wise representation into the multiset attention is theoretically interesting. The integration of this multiset attention into an MBC processing, is practically relevant. The experimental results are convincing, the performance is good across tasks, the stability w.r.t mini-batch size is good.", "weaknesses": "The results and discussion read as if there are only advantages of the proposed method. What are limitations, where does UST not perform well? There is a discussion section, but it does not reflect on the possible disadvantages of UST.\n\nMultiset attention is here posed as a completely new thing. In \"Unlocking Slot Attention by Changing Optimal Transport Costs\" [1] there is also attention across the elements in the multiset. The reference is missing in the related work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5YZvFt68nl", "reviewer": "Reviewer_3i4p", "summary": "The paper proposed a novel trainable aggregation function for sets, called Neuralized Kolmogorov Mean, and a quasi-arithmetic neural networks which includes the Neuralized Kolmogorov Mean in it.  The authors explore both unary and binary (similar to set attention) QANNs. The main contributions of the paper are:\n1. A novel aggregation function for sets - the Neuralized Kolmogorov Mean.\n2. Theoretical justification of its soundness, including proofs of permutation invariance and universal approximation properties.\n3. The results show some consistent although sometimes slight benefit over other methods, with the results about transfer learning to and from classification tasks helping to support the author’s claims of better structured latent spaces.", "questions": "The methodological and novelty part of the paper is convincing, but unfortunately the experimental sections is somewhat weak. I would like to mention that there are many methods in the literature, and it would be unrealistic to expect any paper to include all possible competitors. However, it is needed to compare against the most relevant and closely related methods, such as [1] and [2].\n\n**Methodology**.\n1. As it stands, the paper seems to claim (see for e.g. the abstract) that the learnable function is entirely novel. This is inaccurate, as other methods such as PNA[1] and LAF[2] (among others) have already introduced learnable aggregation functions to address the problem of having fixed aggregation functions. Why did the authors not position their proposed aggregation function in relation to [1] and [2]?\n\n**Experiments**. \n1. Comparisons with closely related works (e.g. [1], and [2]) are missing. Is there a justification as of why those methods were not included in the experimental evaluation?\n2. The reported accuracies for DeepSets[3] are much lower compared to the original work. The paper reports ~65% (see Table 5), while the original paper achieved 82% for 100 points and 90% for 5,000 points. Why is there such a discrepancy? Also comparisons agains other methods are missing as previously mentioned. What is the rationale for their exclusion?\n3. The authors mention a potential application for aggregations is in graph learning tasks. There is quite some literature on neural aggregation for graph level tasks (in the context of combing node representations after message passing) and use of e.g. set transformers in that context can give performance uplift. I see no reason why this method would not work in that context, and I would encourage the authors to investigate this aspect, at least briefly. The large number of practical tasks that have graph representations would help increase the potential impact of this work.\n\n**Results**.\n1. The MNIST-Sets experiments are quite related to those presented in [2]. Why was this comparison not included?\n2. The results for the MNIST-Sets experiments (Table 5, \"sum\" task) are difficult to interpret relative to DeepSets, which reported an accuracy (not MSE) of over 60% for sets containing 50 images. It is unclear how MSE is as a meaningful metric for this task, since the aggregation output (a sum) is an integer. What would the accuracy be for QUANN-1 and QUANN-2 on the sum task?\n3. It is not clear why ablation 2 performs so much worse than Ablation 1, when it is the same as Ablation1 but includes more parameters? Is there any explanation?\n\n**Minor issues**.\n1. Table 5 is too small and it should be broken perhaps in two parts. Experiments from the different subsections are summarized in the same Table which makes the reading a bit confusing.\n2. Research questions section: does the proposed approach [learn] a.\n3. Missing closing parenthesis at the end of the Sum decomposition section\n\n[1] Corso, Gabriele, et al. \"Principal neighbourhood aggregation for graph nets.\" Advances in neural information processing systems 33 (2020): 13260-13271.\n\n[2] Pellegrini, Giovanni, et al. \"Learning Aggregation Functions.\" IJCAI. International Joint Conferences on Artificial Intelligence Organization, 2021.\n\n[3] Zaheer, Manzil, et al. \"Deep sets.\" Advances in neural information processing systems 30 (2017).", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The main strengths of the paper include:\n- the idea is quite simple and easily explained, and the paper is clear with its intentions, methods and outcomes.\n- a novel learnable aggregations function\n- theoretical justifications", "weaknesses": "The mean weakness of the paper are:\n- poor comparison against other learnable aggregation functions\n- the majority of the experiments relies on semi-synthetic datasets and very simple datasets such as Omniglot and MNIST, so there is no evidence that the benefits observed would translate into a more meaningful task.\n- the accuracy on ModelNet40 for DeepSets is quite low compared to the original paper (82% for 100 points, and 90% for 5,000 points), but the authors report ~65% (see Table 5).\n\nAll the weakness are expanded and further explained in the question section below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)