Amortized Mixing Coupling Processes for Clustering
Authors: Huafeng Liu
Liping Jing
Venue: NeurIPS 2022 Accept (2022)
Keywords: generative model
amortized clustering

Abstract: Considering the ever-increasing scale of data, which may contain tens of thousands of data points or complicated latent structures, the issue of scalability and algorithmic efficiency becomes of vital importance for clustering. In this paper, we propose cluster-wise amortized mixing coupling processes (AMCP), which is able to achieve efficient amortized clustering in a well-defined non-parametric Bayesian posterior. Specifically, AMCP learns clusters sequentially with the aid of the proposed intra-cluster mixing (IntraCM) and inter-cluster coupling (InterCC) strategies, which investigate the relationship between data points and reference distribution in a linear optimal transport mixing view, and coupling the unassigned set and assigned set to generate new cluster. IntraCM and InterCC avoid pairwise calculation of distances between clusters and reduce the computational complexity from quadratic to linear in the current number of clusters. Furthermore, cluster-wise sequential process is able to improve the quick adaptation ability for the next cluster generation. In this case, AMCP simultaneously learns what makes a cluster, how to group data points into clusters, and how to adaptively control the number of clusters. To illustrate the superiority of the proposed method, we perform experiments on both synthetic data and real-world data in terms of clustering performance and computational efficiency. The source code is available at https://github.com/HuafengHK/AMCP.

Amortized Mixing Coupling Processes for Clustering
Huafeng Liu Liuh? Liping Jing'
Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University, Beijing, China "The Department of Mathematics, The University of HongKong, Hong Kong SAR, China huafeng wwwwoutlook. com, ipjingebjtu. langgbjt..edu.coo edu. cn
Abstract
Considering the ever-increasing scale of data, which may contain tens of thou- sands data points or complicated latent structures, the issue of scalability and algorithmic efficiency becomes of vital importance for clustering. In this paper, we propose cluster-wist wise amortized mixing coupling processes (AMCP), which is able to achieve efficient amortized clustering well-defined non con-parametric Bayesian posterior. Specifically, AMCP learns clusters sequentially with the aid the proposed intra-cluster mixing (IntraCM) and inter-cluster coupling (InterCC) strategies, which investigate the relationship between data points and reference dis- tribution in linear optimal transport mixing view, and coupling the unassigned set and assigned set to generate new cluster. IntraCM and InterCC avoid pairwise caleulation of distances between clusters and reduce the computational complexity from quadratic to linear in the current number of clusters. Furthermore, cluster- wise sequential process is able improve the quick adaptation ability for the next cluster generation. In this case, AMCP simultancously learns what makes a cluster, how to group data points into clusters, and how adaptively control the number of clusters. To illustrate the superiority of the proposed method, we per- form experiments on both synthetic data and real-world data in terms of clustering performance and computational efficiency.
Introduction
Clustering aims to group samples into certain number of clusters, such that similar samples belong the same cluster, while dissimilar ones to different clusters. Aside from its usefulness in many downstream tasks, clustering is an important but challenging topic which has seen applications in various settings including social-media recommendation [[32], customer partitioning [R]] discovering social networks [[]] and partitioning protein-protein interaction networks [3]. Among various clus- tering algorithms [B77], probabilistic generative clustering models [39] have been widely concerned because of its flexibility and interpretability.
Probabilistic generative clustering models (as known as, mixture models) are staple of statistical modeling in which a discrete latent variable is introduced for each observation, indicating its mixture component identity. These generative clustering models can be roughly divided into two categories: finite mixture model [[]] and infinite mixture model [31]]. In recent years, finite mixture models have been increasingly applied in unsupervised learning problems with the aid of deep neural networks. Instead of using an arbitrary prior for the latent variable, these methods adopted finite mixture prior, such as Gaussian mixture model (GMM) [14, RS). A finite mixture model with a fixed number of clusters may fit the given dataset well, while it may be sub-optimal to use the same number of clus- ters if more data comes under a slightly changed distribution [3I]] It would be ideal if the clustering models can figure out the unknown number of clusters automatically. Alternatively, infinite mixture model the application of nonparametric Bayesian techniques to mixture modeling, which allows for the automatic determination of appropriate number of mixture components. The prior dis-
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

tribution can be specified in terms of a data point sequential process called Dirichlet process (e.g.. Chinese restaurant process (291) where the number of clusters can arbitrarily grow bet- ter accommodate data as needed [2]] However, the inference of nonparametric posterior largely rely on the sequential data point modeling (e-g., Markov Chain Monte Carlo (MCMC)), which is time-consuming, with convergence that is difficult to assess.
While above probabilistic clustering generally approached an unsupervised learning problem, the amortization power of non-linear transformation (e.g., neural networks) has recently spurred progress on supervised formulations, based on novel tractable objective functions for generic data and an arbitrary number of clusters [20. 211 27] In exchange for the additional cost of procuring labeled training data and training the model, amortized clustering offers several benefits on top of time efficiency at test time. Traditional unsupervised clustering, especially probabilistic clustering, defines specifie likelihood and prior on the data geometry, even without any prior knowledge of the data, which are not needed with amortized clustering, since the model learns based on the statistics of the labeled training data [2I]]. Moreover, to achieve the best result (e-g.. agreement with human labeling with a well-defined posterior), unsupervised methods usually need parameter tuning and post-processing (e.g., cluster merging) [[S), which are obviated with amortized inference by implicitly learning the definition of clusters underlying the training datasets, In sense this shares a similar philosophy as Neural Processes [Mm. g. [[]]. which learns from multiple datasets to learn a prior over functions in meta learning manner.
In this paper, motivated by these prior work, we propose cluster-wise Amortized Mixing Coupling Processes (AMCP), which is able to provides an efficient clustering architecture by grouping data points in an optimal transport cluster-wise view. each cluster-wise step, AMCP generates clus- ter sequentially and finds one cluster a time. In this case, two key strategies, Intra-Cluster Mixing (IntraCM) and Inter-Cluster Coupling (InterCC), are proposed to measure the relations between different data points and generate cluster. Specifically, IntraCM focus investigating the relation- ship between data points and reference distribution a linear optimal transport view [{3]]] which avoid pairwise calculation of distances between clusters and reduce the computational complexity from quadratic to linear in the current number of clusters. Then, InterCC leverages new insights on defining dissimilarity between assigned data points and the unassigned data points based on the couplings obtained in IntraCM, which is able to select the dissimilar data points to form the next cluster and update existing clusters. The key contributions of our work The proposed AMCP not only inherent the efficient cluster-wise learning manner, but also reduce the quadratic cost of traditional similarity weight computation with the aid of optimal transport mix- ing. (2) Novelty. To the best of our knowledge, AMCP is the first to introduce optimal transport for amortized clustering and effectively correlate the relationship between optimal transport and mixture models. (3) Adaptivity. Non-parametric learning manners exist not only in model parameter learning but also in cluster generation, which allows unsupervised parameter learning and non-handcrafted intervention. (4) Flexibility. Different from existing amortized clustering methods without changing the rrready-generatee clusters. AMCP is able to dynamically adjust the previous generated clusters to avoid the accumulated errors with the aid of mixing and coupling manners. (5) Effectiveness. Extensive experiments conducted on both synthetic and real-world datasets demonstrate that AMCP can cluster data effectively and efficiently.
follows: (1) Efficiency.
2 Related Work
In this section, we briefly introduce some recent developments in several related topies, namely probabilistic clustering and amortized clustering.
Probabilistic Clustering Probabilistic clustering models (or equivalently, mixture models) are a sta- ple of statistical modeling in which a discrete latent variable is introduced for each observation, indi- cating its mixture component identity. There is growing interest in developing probabilistic cluster- ing methods using parametric or nonparametric prior for complex data modeling [R,, 31,, 14, 38], and finite and infinite mixture models were developed. The main focus of finite model introduces finite mixture prior with fixed number of cluster to clustering. In this case, Gaussian distribution or non Gaussian distributions [24]] [X] are widely used to model data distribution [6]] Alternately, infinite mixture models [2] adopt non-parametric prior to control the number of clusters dynamically, such
as Dirichlet process and its variants [2%]. Recently, researchers focus on deep generative clustering
by combining probabilistic clustering modeling and neural networks [|
27 Mi
Ml 36, E]. The

---

main focus of these methods is to learn representation of input data amenable to clustering via deep neural networks.
Amortized Clustering Clustering is generally formulated as an unsupervised learning manner and while amortized clustering [27, 20. 21] investigates the amortized power steam from a predefined supervised formulation to achieve an arbitrary number of clusters in new task. Lee et al. [20, 21] adopt attention networks output instead the cluster labels of each data point, for general mixture model, with an arbitrary number K of clusters and a cost of O(K) forward passes. Later, Pakman al. [2-]] presented Neural Clustering Process (NCP), which sequentially computes the conditional probability of assigning the current data point to one of already constructed clusters or new one. The most similar work to our methods is NCP [27]] and CHiGac [25].. Both of them explain cluster- ing from the perspective of generative models, which similar in spirit to the popular Gibbs sampling algorithm for Dirichlet process mixture models, but without positing particular priors on partitions. Besides, NCP and CHiGac rely heavily on the selecting of anchor points and the processing ordering and often exhibits unstable properties. Different from previous work, AMCP connects the relation- ship of mixture model and optimal transport to describe the cluster's summary statistics well, and seamlessly combines mixing and coupling to achieve efficient amortized clustering.
Method
3.1 Notations and Problem Formulation
Given N data points D = {xi, Xa, Xxh. where x, E R indicates feature representation for the i-th data point represented in d-dim latent space. Probabilistic models for clustering is usually presented by sequential sampling procedure to generate clusters. Usually, the generative process is & ~ p(es (n). μ4 ~ pa2) X, ~ p(x: (.:)) here e; E {1.2. K} encodes the cluster assign- ment of the data point Xj. 01 and o2 are hyperparameters. The integer random variable K indicates the number of distinct values among the sampled Gj, and l4 denotes parameter controlling the distribution of the k-th cluster. Specifically, these models include Gaussian mixture model [14, [8],
Dirichlet process mixture model [X] and etc.
Point-wise Clustering Posterior Probabilistic model for clustering is usually presented by sequen- tial sampling procedure to generate clusters. One of the prototypical tools for nonparametric cluster- ing modeling is the Dirichlet Process. It allows for a discrete distribution of observations drawn from an arbitrary base measure over the domain in such a way as that the marginals match draws from, while simultaneously obtaining a countable set of distinct data points. A useful view of the Dirichlet process mixture model the Chinese restaurant metaphor [122] which sequentially computes the conditional probability of assigning the current data point to one of already constructed clusters or a new one, The cluster assignment for each data point can be sampled by a ~ p(c: ((= DD) Here C1--1 {c1, C2, 3--13 indicates the existing clusters formed by already assigned data points {xI, X2. .. x.-1). Based on existing clusters 1 and observations D, the cluster assignment for the i-th data point is sampled.
Given N data points X (x.)),., it natural to draw independent samples form clusters using decomposition
p(C1.N [D) p(e1 (D)p(ezles, D) ...... www../eeeeeee-.-e.... D)
After all data points are assigned, the final clusters are formed. Inference proceeds by traversing data points and re-sampling their cluster assignments in sequence.
Challenges The above process sequentially computes the conditional probability of assigning the current data point to one of already constructed clusters or a new one, and does not posit any par- ticular prior on partitions. The sequential sampling procedure makes it process data points one by one, which limits its scalability on large datasets (the computational cost for full i..d. sampling {ci, C..... CN } is high) [[T]] Furthermore, the clustering result is sensitive the sequential pro- cessing order. It needs sufficient number of random samples to obtain stable result, which is time-consuming.
3.2 Amortized Mixing Coupling Processes
For the seek of effective cluster generative process and preventing the effect of sequential processing order, we focus on an alternative of clustering generative process from the view of cluster rather than the view of data point 2 [
Let C (Ci,C2,... ,Cx) indicate K clusters, define Ca
3

---

as C. = (ch c. ..... C...).)))). where ck is the i-th data point of the k-th cluster. N& is the number of data points belonging to the k-th cluster. Based on the above definition, we are interested in sampling each cluster assignment by Cs ~ p(C A ,,... K, where Sy indicates the conditional information containing Mr available unassigned data points for constructing the k-th cluster. M2 is the number of left data points after generating the previous k - 1 clusters, defined by M₁ - N - - N₂ (N, is the number of data points belonging to the j-th cluster).
We formulate cluster assignment process in the form of cluster-wise (C2 + C2 -... - Ck) rather than point-wise (ei -> 02 - - ex), which enables an efficient clustering process. We are interested in sampling [C1, Cr ... ,C] via a decomposition as follows:
PDC =
p(C (I.)
which is easy to model cluster sequentially, and the arbitrary number of clusters can be adaptively determined until there is no remaining point in Sg. The sampling process defined in Eq.(2) indicates a clustering posterior using K sets of indices.
To model the above cluster-wise sequential posterior, we aropose a cluster-wise amortized mix- ing coupling processes (AMCP), which is able to provides an efficient amortized clustering ar- chitecture by grouping data points in a cluster-wise view rather than point-wise view. As shown in Figure (a(a), AMCP generates clusters scquentially and two key strategies, Intra-Cluster Mix- ing (IntraCM) and Cluster Coupling (InterCC), are proposed. Specifically, as shown in Fig- ure I1, IntraCM focus on investigating the relationship between data points and reference distribution (b..T) = [ (b.))) 22 (04422)
.kk ( with 3/!-3) Ra in linear optimal transport view [63]] with T inner update steps. Then, InterCC leverages new insights on defining dissimilarity between unas- signed data points in S₁ and assigned data points in C2,, Ca.... CC--1 } based on the learned optimal transport plan Q (R::7) obtained in IntraCM in a coupling manner, which is able to select the dissimilar data points form the next cluster Cj and update existing clusters {C1, C2. CCx1)
3.2.1 Intra-Cluster Mixing
We present intra-cluster mixing (IntraCM) from generative mixing view, novel and fast framework for embedding each cluster in a vector space and defining the relations of each data point from the reference distribution, which give clear evidence or constraint that the learned distributions could describe the cluster's summary statistics well, as shown in Figure I[b)). Specifically, cluster generation step, data points are partitioned into several subsets, i.e., DDk) has structures with (C1, C.. C,-1, S. We model each data point x, E (Dk) as a Bayesian mixture model with component parametrized by reference vectors & (k) = =00 o2 , &6
the k-th
E Rexd and non-fixed mixing proportions over k: - 1 clusters {C1, C2,-.. _C--1) and unassigned data points Sa, as shown in Figure Ilaa) differentiable non-linear function se(() (a neural network) is used to transform these representations /0 into parameters (1)) By introducing latent variable o(t) indicating the data point x; comes from the j-th component, can get the full mixing likelihood related to the k-th cluster generation as follows:
,....
(( c (k)
II DD
Σ
p (xx (1o((()))) (R)
II DON
p
).
(3)
where b(() are the mixing coefficients.
,ba) encodes the data point assignments. π (k) (k)
(k)
12
However, directly optimizing log II p(x, (o(((())))) with respect to Φ is difficult due to marginalization over b(²), while for many distributions optimizing log II p(x;, b((a) (oo())))) is much easier. In order to reflect the relationship of data in different sets, we introduce a variational distribution 4:. q(b_)) (x.) indicating a variational probability that the data point x; belongs to the j-th component. In this case, we introduce a constrained variational lower bound to satisfy the optimal transport measure among different components and focus on performing the expected log likelihood by optimizing a lower bound:
P xx, 6,3 (1(
Σ log ((k)
Σ 1=1
p x., ( 1o)
Σ
((b x.) log
700() (x:)
(4)
Diki ,=1
=: ((b Q (1))

---

(((-1))) IntraCM
@ I
p IntraCM 2..
G
Cx2
St-1
C2-1
St.
Sp41
c: Cu
p
0
E: a
E: 0
M:
M.g(h (1)
of
Ce-1 Intercc 100 Ca-1
InterCC(g/err)
un InterCCr (m1m))
Cz+1
(a) The architecture of the proposed cluster-wise AMCP including two key strategies: Intra-cluster mixing (In- traCM) and inter-cluster coupling (InterCC). IntraCM and InterCC performs to generate cluster one by one.
Gx
(
p. 407 IntraCM(G ..) G. (n
Ins Cluster Dtiding
liser Qustar Goupling
(b) Intra-cluster mixing and inter-cluster coupling. (c) Optimal transport view of intra-cluster mixing Among them, IntraCM cooperates optimal transport and inter-eluster coupling, which focus on embedding from mixture view and describe the cluster's 's summary each cluster in a vector space and defining the rela- statistics with the aid of multiple EM steps. Then, In- tions of cach data points from the reference distribu- terCC regarding the output of IntraCM as coupling to tion.
generate the next cluster.
Figure 1: The architecture of the proposed AMCP.
Optimal Transport View of IntraCM Here we give a detailed analysis to connect the proposed mix- ing strategy with optimal transport measure [33], III [Z],, which defines the transport plan between data points as weighted points clouds or discrete measures. We aim to formulate an optimization problem for q(bl (x.) given in (9). By replacing the variational distribution to a joint distribution, i... 9h3 q(bb.) x (xx),, the optimization problem of maximizing the lower bound in (q) with
(k)
respect the transport plan matrix Q (a E R Nxk equivalent to solving following entropic regularized Kantorovich relaxation of optimal transport [28]:
J (k)
(6)
min (
ΣΣ
- log x D (o)()()))) 2.3 - (Q (k)).
(5)
D(k)
where - p(,, b.j Ifo ())) indicates the pairwise transport cost between data point and reference
vector. If Gaussian distribution is used as data likelihood, the transport cost - log p(x, b[,) ())) is equivalent to a negative kernel -K((,,, fo(()"))) The impacted entropic regularized term (IQQ)) Ex,, R.j log 1.g controls the sparsity of (()).
(k)
In original entropic regularized Kantorovich relaxation optimal transport [28]] there are some constraints on transport plan ((),, i.e. ΣN 4 = 1/a and Σ (8) 1/N. we set the factorized
variational distribution to q(x:) = 1/x, the constraint E, 4 1/x is automatically met. Optimal transport view induces various advantages of our proposed mixing strategy. First, it is computa- tionally efficient, scalable to vary large dataset. Unlike the quadratic cost of traditional similarity weight computation (e.g., the attention score in self-attention). our optimal transport mixing uses a fixed number (k) of references serving queries and the number of references is set depends on data without manual setting. Secondly, it allows unsupervised learning where the model parameters {o)
1 1==1 can learned without target (class) labels. Third, it is able to sufficiently exploit pairwise interactions between data points in intra-cluster level and benefit for the next inter-cluster coupling, which is useful to sufficiently mine the hidden structure among data.
Expectation Maximization for IntraCM Considering the latent variable (k) is introduced, natu- rally the expectation maximization (EM) method can be utilized [[]]]. Iterative optimization of this bound alternates between steps:
E-step: The E-step focuses on maximizing the above bound with respect to the variational prob- ability g(b(5.0)) [x.). We compute new estimate of the posterior probability distribution over the
5

---

latent variables from the previous iteration, which yields a soft-assignment of the data points to 2 fa. p (x (o 1), Jo(00(t
the components g(b (1.0))
(x.)
=5 fh. (x. 1o. (D( where x; (((ht-1) and (((:11)) in-
dicates model parameters from previous iteration. This voft-asiigmment is regular E-step without the balanced assignment constraints. Furthermore, the variational lower bound can be formulated as the optimal transport setting (as shown in (B)())) yielding an Sinkhorn-baas -aaoed optimal transport solution [B0] as follows:
albi (x.) u4 exp (xx10(5 fo((5th.t)1))) /e) vj.
exp
(6)
where u; 3= exp(x(x,1b],, So D)3/sc)u; and Uj S. exp[p(x; 18! 1) (oo((((..111)))//0
Note that above solution usually converges quickly after few iterations. Hence, the E-step coin- cides with the optimal transport based similarity weight computation, which leads to the various advantages of our proposed mixing strategy and forms the transport plan Q for all data points.
M-step: Once the E-step done (i.e., Q (h.t) is found), we perform the M-step, that is, maximiz-- ing the lower bound in Eq. (E) with respect to ((t.t) when ((*.t) is fixed. Due to the non-linearity of (-) there exists no analytical solution to arg maxg(0) ((b ((h.))). With the benefit of dif- ferentiable (.), we can solve it with the aid of gradient-based strategy. Assume that likelihood
p(x. ( (b)),, b,.)) follows a Gaussian distribution with fixed covariance, we can obtain following update rule:
$ (h.0)
$ ( +1 Σ
(k.t) (00,t (xi) (fo(o((t-1))- -xi) Vielof
D(K)
For simplicity, if we assume the parameters (k)) in likelihood with linear transformation (o() ( it can yield an optimal transport kernel embedding solution [26] when we
(k.1) Σ, 00 (0..,..)pp)))) (x)) and (/h000
set it as Gaussian distribution with fixed variance, c.g., my
(x (Wx:)/vl) Wx; which is equivalent to induced self-attention introduced Lee
Σ E exple (1) (Wx)/vā)
al. [20] providing an efficient solution for pairwise weights measuring.
Usually, we can take multiple EM steps for further optimization (e.g., (0t00) + Φ (A.1) + + s (K.T). Here T is the number of EM steps. IntraCM belongs the class of generalized EM algorithms and is guaranteed to converge to a local optimum of the data log likelihood [5)]
3.2.2 Inter-Cluster Coupling
After T EM steps, the variational probability (. = -(b, q(b,t, (x,) can be regarded as the optimal transport plan, which carries the information on how distribute the mass of data points x, to reference vectors o (4,T)) By regarding (((,7) couplings between unassigned set and assigned set, we propose inter-cluster coupling (InterCC) to generate the next cluster, as shown in Figure M. Considering the current data points have structures with & different sets, C1, C2, Ca-1, can } related to k existing clusters and ((k) get k transport plan matrices (T(()) U ,... -..),
related to the unassigned set Sk.
( x, Ci) Cylxk 1= (1.2.....)) and ( = [[(bb)))
Mxx
(8)
We can measure the relations between unsigned data points in S₁ and existing assigned data points in k 1 existing clusters C2,C2.... CC-11 with the aid of couplings (0() as follows
M (
(U (1.T) (b)
(U(+)
(4.27
(A) Mi
RRNN-M
where + is a column-wise concatenation operation and v able to select dissimilar data point in unassigned data points set Sy to form new cluster Ct, i.c.
(k) is the i-th column of (*), It is reason-
Ca ~ MULT SOFTMMX ([-- m 1- m .1.,,1 ,1 I (1) 1mmm))
(10)
Here Multinomial distribution is used to model the generation of Cr. E N- is vector of whose elements are all 1s. m, (8) is the i-th column of (k), Thus, the data points related to dissimilar

---

indices in each matrix are selected to generate the new cluster Cg. Furthermore, previous existing clusters Ci, Ca, ... CC--1 can be updated based on dissimilarity matrix. For a given cluster Cı, it can be updated by C == Ci GT where Ci := MULr(SOFTMAX(1 - M(..))). I E R a vector of all Is and ((..)) U. (()
3.2.3 Iterative Unrolling Inference
IntraCM and InterCC are both differentiable procedures for optimal transport measuring and next cluster constructing, whose outcome relies on the current existing clusters and the statistical model parameters (k)), As shown in (Q) the whole cluster-wise clustering procedure can be regarded as a recurrent IntraCM with InterCC until there are no available data point left. In this case, the information about statistical regularities in our model that is required for generating the sequential clusters encoded in a series of parameters ( <24.2) .... . (I..T)). By unrolling the itera- tions of the presented mixing process, we obtain an end-to-end differentiable clustering procedure based on the statistical model parameterized by (6)0.)) gb2,7),.. ,... ((K,)). We can therefore use (stochastic) gradient descent and fit the statistical model to capture the regularities corresponding to clusters for given dataset.
K Σ
(xi (((
Epp wo EptC 2))
Σ
4(05)) (ei log
p(Cr (a(
((b ((,))(x)))) x.)
InterC
IntroCM
(11)
The likelihood is derived from approximating the po (C to true posterior via KL divergence.
Note that samples from p(Cu.x, D) are obtained from the explicit generative model, such as Gaussian mixture model, Dirichlet process mixture model. With infinite generated samples, model to approximate (C(Cu:k (D) accurately. The whole inference procedure is given in Algorithm By repeating the cluster generative process until there are no data points left, IntraCM and InterCC are processed alternatively, and the clusters are iteratively refined by deriving the explicit supervisory signal from the already formed clusters. Note that the complexity of traditional methods measuring weights between data points acquires O(K 2²) (e.g.. self-attention) which may be too expensive for large sets, and our AMCP only acquires N(KNk) in the EM-step for the k-th iteration, which more efficient.
can train
Algorithm Iterative Unrolling Inference for for AMCP
Input: D = (X1, x-. } with N data points. Randomly initialized parameters (Φ (5.0)) K I: Establish the first cluster Ci by selecting cluster with the highest prohability from k-means or GMM
2: while not new cluster C₁ is generated do
3: Intra-Cluster Mixing
4: Establish expected lower bound (k. Q ((..0)) via Eq.
5: for = 1,2, .'' do
Multiple EX steps
6: Qth (kd)))) < (( 13 Q(he-1)), (k,,0 +
(( Qlh!i)) b EM-step
7: end for
8: // Inter-Cluster Coupling
9:
Establish & transport plan matrices {U (1()) M=1 and (*),, and similarity matrix (R)
10:
Generate C.
11: Update existing clusters C1, C2, "a,, C
Optional
12: Sr + (1 then
13:
Establish unassigned set S
14:
end if
15:
end while
Output: updated parameters (K,,), generated clusters C2,C2, C2....,..
4 Experiments
In this section, we evaluate the proposed AMCP in terms of synthetic and real-world datasets by comparing with the state-of-the-an methods.
4.1
Experiments on Synthetic Datasets
The first experiment was conducted on synthetic data with arbitrary number of clusters. Thus, five existing methods handling variable number of clusters, MCMC[M), vIø], DAC[TT], NCP[2Z] and
7

---

Table 1: Clustering performance on synthetic 2D GMM. The numbers below Scenario
and 2 are
oracle LL values computed by the true parameters. The average on 5 runs are reported.
Secnario
Metrie
MCMC [
DAC [21
NCP [AM]
ST-ACT [[X]]
AMCP
LL
1.5011
-1.4230
-1.2671
-1.3251
-1.2928
00001
-1.2517+0.085
Scenario / (-1.2452)
ACC
0.706620.003
0.683220.001
0.792320.001
0..10100.002 40.002
0.8131
20.003
20.002
MMI
0.873120.00 20.004
0.8531
2.8083
0.8923
20.004
0.9123-20.00
20.001
Time[s]
0.083220.09
0.03321u.040
0.0124
0,0343
a. 026
0.0114 10.000
0.0114
0.0113
003
-1.92410.022
2.9902 22.9026_..022 2.0022
10.44400.044
-1.855T....0 042
-1,8918
0.005
-1.8420100002
-1,3420
10.023
Scenario 2 (-1.8242)
ACC
0.6531 2o.0n2
0.5944.0..o3
0.7021
20,004
0.7233
0.6926.n.002 2.0002
0.6926
0.7574
MMI
0.7931
0.8123
0.8231
0.8231 #0,002
0.8089
20.004
2.00002
20,002
Time[s]
0.0953
0.0362::0.06
20.004
0.0366g0.035
0.0166:n.001
0.0124
ST-ACT(20] are used as baselines. Among them, MCMC and VI are traditional approximation methods, and DAC, NCP and ST-ACT are deep amortized clustering methods. The synthetic data generated via a 2D Gaussian mixture model (GMM) which is defined by the following process ~ ExP(1), C1:N ~ CRP(a), f4g ~ NN0,, N(0. of,1,, x; ~ -N(pe I²I). We set n₁ = 10. At each training step, we generate 10 random datasets according to the above generative process. Each dataset contains 200 points on a 2D plane, and each sampled from one of 4 Gaussians. Two testing scenarios (Scenario and Scenario 2) are constructed to evaluate the effectiveness of the proposed method. The test set in Scenario 1 has the same configuration (200 samples and 4 clusters) as training set, while Scenario 2 contains different numbers of samples and clusters (400 samples and 6 clusters) in order to verify whether the clustering method can generalize to the unseen clusters. Table M1 summarizes the results in terms of two testing scenarios. Here we report log-likelihood (L.., clustering accuracy (ACC), Normalized Mutual Information (NM/) and running time (seconds). The best and second results are marked in bold and underlined. As expected, the proposed AMCP con- sistently outperforms baselines on both scenarios. Although Scenario 2 is more challenging, amor- tized clustering methods (DAC, NCP, ST-ACT and our AMCP) can capture cluster uncertainty and obtain competitive results. For testing time, amortized methods are obviously faster than traditional sampling based method (MCMC). Although VI is much fast than MCMC, it is still lower than amor- tized methods. In cluster-wise amortized clustering methods, DAC, ST-ACT and our ACMP achieve competitive running time since cluster-wise sequential modeling is more efficient than point-wise sequential modeling.
4.2 Experiments on Real-world Datasets
Real- datasets, MNIST [[ Tiny-maggeNet [ and CIFAR- 10 G are used to validate the performance. To formulate the amortized mechanism, we generate training and test data via fol- lowing Dirichlet process mixture model: a ~ Exp(1), CINN ~ CRP(a), K - ~ BINOMIAL 1,0.5), 4 U10, K - 1), ~ le.. Here x; UD, Iz, indicates sampling x; with label le uni- formly from dataset D. which can be training set or test set. K, indicates the number of clusters we can sample. For all original dataset, data containing half of the classes is used to generate the training sets, and the remaining half is used to generate the test sets, with no overlap between training classes and test classes. We sample multiple clustered datasets to form training sets with N {1000, 10000}. For test set, we generate 1000 randomly clustered datasets with N E (1000, 5000]. More information is given in Appendix.
In Table 2 we depict the quantitative clustering results in real-world datasets. The best and sec- ond results are marked in bold and underlined. The average clustering results are recorded over runs with different random parameter initialization. It can be seen that our AMCP outperforms all baselines. The main reason, we believe, is that AMCP not only exploits amortized properties between data points in both inter- and intra-clusters, but also has flexible clustering process, which simultaneously generate the new cluster and update the summary statistics for previous clusters. In model efficiency evaluation, the DAC, ST-ACT and AMCP consume a more competitive running time, which outperform other methods by large margin. The reason is that DAC and ST-ACT iden- tify each cluster after one forward pass, and AMCP generates clusters in cluster-wise manner and exploits data relations via the efficient optimal transport. However, DAC, ST-ACT are pretty worse than AMCP on clustering performance.
Among amortized clustering methods, DAC, NCP, ST-ACT and AMCP are able to process arbitrary number of clusters. For better visualization of cluster generative process, a digit subset from MNIST test set is randomly sampled. To illustrate how the clustering models capture the shape ambiguity
8

---

Table 2: Clustering performance real-world datasets.
The average
5 runs reported.
Datasel
Metric
MCMC [0]
V[
DAC [20]]
NCP [2
ST-ACT [20]
AMCP
ACC
0.86300_0.004
0,9758 20.004
0.9796-0.00
0.9633
0.959620.003
0.9845
=0.001
MNIST
NMI
0.912410.005
0,9482 10.000
0.9542 10.001
0.9321 10.002
0,9465
0.9624
.0.001
Timels]
56.33+6.42
63,4646.42
24.33+3.58
73.3442.45
22.56+2.57
18.22
ACC
0.0637=0.005
0.0785=0.001
0.0907=2.0004
0.1028
0.0892=0.003
0,1324
Tiny-
NMI
0.3212-20.02
0.323 200004
0.2894
0.3254 2000022
0.2832..0.00
0.3421 =0.000
ImageNet
Time|s]
3140.8 26
3450.5
1415.6g10.57
4732.8g33..1
1316.6g21.15
1186.4
1866.4_10.30 19:37
ACC
0.6322:20.0o
0.8742.20.002
0.883520.00s
0.7641 =:0005
0.863222..0z
0,9024
122.4400.00 0.00
CIFAR-10
NMI
0.6782.0.004
0.7765
0.7873g0.002
0.7431
0.7673=0.005
0.8042 -00001
Time[s]
174.42
192.42
78.33
239.33
68.32
59.33
Table A toy clustering results comparisons among different amortized clustering methods (DAC, NCP, CHiGac and AMCPP on MNIST dataset.
Ground-murh:
0
DAC:
NCP:
0 S
CHiCie:
AMCP:
of some of the digits, we plot the ground-truth digit clusters and clustering results in Table B. Com- paring with DAC, NCP and ST-ACT, the proposed AMCP get the most accurate results. For example, DAC assigns the digit 7 (with similar appearance to 9) to cluster 9, and NCP generates a new cluster for it. Fortunately, AMCP correctly assigns it to cluster 7. All baselines unable to assign the digit& 4 written a strange way to the right cluster, and our AMCP correctly assigns it to cluster 4. Results on Neny-ImageNet and CIFAR-10 can be obtained in Appendix.
4.3 Model Analysis
We empirically analyze the proposed model from the impact of various summarized follows: (1)The Number of EM Steps T: AMCP can apply multiple EM steps to achieve more accurate &f (K.T)) Here we conduct experiments on varying T in 3.4,5 to investigate the effect of T. Figure (((a) and (b) show the effect of on synthetic data (Scenario and real-world data (CIFAR- 10). We can see that, the performance (L. and ACC) become better and better with the increasing of T, while the running time linearly scalable to the number of steps. (2) OT E-step vs. Regular E-step We introduce two different ways to conduct E-step in IntraCM: regular soft-assignment E-step and Sinkhorn-based optimal transport E-step. Figure (ic) and (d) show the comparison between regular E-step and OT E-step on synthetic data (Scenario 1) and real- world data (CIFAR-10) The maximum number of Sinkhorn-Knon problem is set to 10. While the regular E-step is consistently faster than the OT E-step since it's closed-form solution, OT E-step can achieve more accurate likelihood and accuracy. Furthermore, compare the difference between the strategy of updating the existing clusters and the strategy of not updating. More empirical results are given in Appendix.
They are
iterations for solving the OT
-12
0.02
0.92
90
-1.22
0.905
-1.25
70
-1.24
0.9
0.01
0.9
0.895
Timels)
Time(s]
-135
0.88
30
4
0.885
regular-E
OT-E
regular-E
OT-E
(a) T T-(Seenario /)
(b) T-(CIFAR-I0)
(c) E-step (Scenario /)
(d) E-step (CIFAR-1O)
Figure 2: The impact of hyper-parameters.
5 Conclusion
In this paper, we studied the problem of amortized clustering, and presented our approach that perform cluster-wise amortized mixing coupling processes via intra-cluster mixing and inter-cluster coupling. An interesting direction for future research is to explore more complicated and realistic

---

hidden structure (e.g,, hierarchical clusters) that is not possible with our approach. Furthermore, it is interested to explore novel applications that can be enabled by the interpretability and controllability brought by the amortized clustering.
6 Acknowledgement
This work was partly supported by the Beijing Natural Science Foundation under Grant (Z180006, L211016); the National Key Research and Development Program (2020AAA0106000); the National Natural Science Foundation of China under Grant 62176020; CAAl-Huawei MindSpore Open Fund; and Chinese Academy of Sciences(OEIP-O--22000)))
References
[1] Christophe Andrieu, Nando Freitas, Arnaud Doucet, and Michael Jordan. An introduction to memc for machine learning. Machine learning, 50(1-2):5-43, 2003.
[2] David M Blei, Michael I Jordan, al. Variational inference for dirichlet process mixtures. Bayesian analysis, 1(1):121-14,, 2006.
[3] Sylvain Brohee and Jacques Van Helden. Evaluation of clustering algorithms for protein- protein interaction networks. BMC bioinformatics, 7(1):1-19, 2006.
[4] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the ECCV, pages 132-149, 2018.
[5] Gianfranco Chicco, Roberto Napoli, and Federico Piglione. Comparisons among clustering techniques for electricity customer classification. IEEE Transactions on power systems, 21(2): 933-940, 2006.
[6] Review By: A. Clifford Cohen. Finite mixture distributions. Technometrics, 24(4):339, 1982.
[7] Aurelien Decelle, Florent Krzakala, Cristopher Moore, and Lenka Zdeborová. Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Physical Review E, 84(6))066106, 2011.
[8] Thomas S Ferguson. A bayesian analysis of some nonparametric problems. The annals of statistics, pages 209-230, 1973.
[9] Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Mur- ray Shanahan, Yee Whye Teh, Danilo Rezende, and S. M. Ali Eslami. Conditional neural processes. In Proceedings of the ICML, pages 1704-1713, 2018.
[10] Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola. Danilo Rezende, SM Es- lami, and Yee Whye Teh. Neural processes. arXiv preprint 10X7vv/807.0166 2018.
[11] Klaus Greff, Sjoerd Van Steenkiste, and Jürgen Schmidhuber. Neural expectation maximiza- tion. Advances in Neural Information Processing Systems, 30, 2017.
[12] Thomas I. Griffiths, Michael I Jordan, Joshua Tenenhaum, and David M Blei. Hierarchical topic models and the nested chinese restaurant process. In Proceedings of the NeurlPS, pages 17-24, 2004.
[13] Hemant Ishwaran and Lancelot F James. Gibbs sampling methods for stick-breaking priors. Journal of the American Statistical Association, 96(453):161-17, 2001.
[14] Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou. Variational deep embedding: An unsupervised and generative approach to clustering. In Proceedings of the IJCAI, page 19651972, 2017.
[15] Uzay Kaymak and Magne Setnes. Fuzzy clustering with volume prototypes and adaptive cluster merging. IEEE Transactions Fuzzy Systems, 1006)775-772,, 2002.
10

---

[16] Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. Attentive neural processes. In Proceedings of the ICLR. 2019.
[17] Minyoung Kim. Differentiable expectation-maxiiizatiii for set representation learning. In International Conference on Learning Representations, 2021.
[18] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7:7, 2015.
[19] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 1):2278-2324 1998.
[20] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention- based permutation-invarian neural networks. In Proceedings of the ICML, pages 3744-3753, 2019.
[21] Juho Lee, Yoonho Lee, and Yee Whye Teh. Deep amortized clustering. arXiv preprint 1XXiv:999.113433 2019.
[22] Xiaopeng Li, Zhourong Chen, Leonard K. M. Poon, and Nevin L. Zhang. Learning latent su- perstructures in variational autoencoders for deep multidimensional clustering, In Proceedings of the ICLR, 2019.
[23] Tsung I Lin, Jack C Lee, and Wan Hsieh. Robust mixture modeling using the skew distribu- tion. Statistics and computing, 17(2):81-92, 2007.
[24] Tsung I Lin, Jack C Lee, and Shu Y Yen. Finite mixture modelling using the skew normal distribution. Statistica Sinica. pages 929,
[25] Huafeng Liu, Jiaqi Wang, and Liping Jing. Cluster-wise hierarchical generative model for deep amortized clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15109-151 2021.
[26] Grégoire Mialon, Dexiong Chen, Alexandre A'ppprmoot,, and Julien Mairal. A trainable optimal transport embedding for feature aggregation and its relationship to attention. arXiv preprint arXiv:2006. 20065, 2021.
[27] Ari Pakman, Yueqi Wang, Catalin Mitelut, JinHyung Lee, and Liam Paninski. Neural cluster- ing processes. In Proceedings of the ICML, pages 1455-7465, 2020.
(28) Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends in Machine Learning. 11(5-6)355-6077 2019.
[29] Jim Pitman et al. Combinatorial stochastic processes. Technical report, Technical Report 621, Dept. Statistics, UC Berkeley, 2002. Lecture notes, 2002.
[30] Richard Sinkhorn and Paul Knopp. Concerning nonnegative matrices and doubly stochastic matrices. Pacific Journal of Mathematics, 21(2):343-34 -48, 1967.
[31] Yee Whye Teh, Michael Jordan, Matthew J Beal, and David M Blei. Hierarchical dirichlet processes, Journal of the american statistical association, 101647)) 2006.
[32] Chih-Fong Tsai and Chihli Hung. Cluster ensembles in collaborative filtering recommendation. Applied Soft Computing, 12(4):1417-1225,000 2012.
[33] Cédric Villani. Optimal transport: old and new, volume 338. Springer, 2009.
[34] Cinzia Viroli and Geoffrey McLachlan. Deep gaussian mixture models. Statistics and com- puting, 29(1):43-51, 2019.
[35] CF Jeff Wu. On the convergence properties of the em algorithm. The Annals of statistics, pages 95-103, 1983.
[36] Junyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In Proceedings of the ICML, pages 478-487, 2016.
11

---

[37] Dongkuan Xu and Yingjie Tian. A comprehensive survey of clustering algorithms. Annals of data science, 2(2):165-193, 2015.
[38] Linxiao Yang, Ngai-Man Cheung, Jiaying Li, and Jun Fang. Deep clustering by gaussian mixture variational autoencoders with graph embedding. In Proceedings of the ICCV, pages 6440 6449, 2019.
[39] Ron Zass and Amnon Shashua. A unifying approach to hard and probabilistic clustering. In Tenth IEEE International Conference Computer Vision (ICCV'05) Volume I, volume 1, pages 294-301. LEEE, 2005.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Abstract and Introduction.
(b) Did you describe the limitations of your work? [Yes] See Conclusion
(C) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [No]
(c) Did you include any assets either in the supplemental material or URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you'te 'r using/curating? [No] All assets used in this paper are open source
(c) Did you discuss whether the data you are using/curating contains personally identifi- able information offensive content? [No] Does not contain personally identifiable information or offensive content
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
12


Reviews:
[{"review_id": "h7VoeCOZSh7", "reviewer": "Reviewer_qZcB", "summary": "In this paper, the authors propose a method called amortized mixing coupling processes for clustering (AMCP). ACMP learns the relationship between samples and reference distribution with intra-cluster mixing (IntraCM); while assigning samples to clusters and generating new clusters with inter-cluster coupling (InterCC). The ACMP method is tested on synthetic and real-world datasets in the experiments.", "questions": "1. What are the contributions of this paper?\n2. What is the assumption of the proposed clustering method? How is it different from existing amortized methods?\n3. Is the objective function defined in Equation (11) guaranteed to converge?\n4. In Table 1, does the time mean training time?\n", "limitations": "The authors did not address the limitations and potential negative social impact of their work.\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe authors propose a new clustering method called AMCP.\n\nThe method outperforms its competitors in terms of clustering performance and computational efficiency.\n\nWeaknesses:\n\nThe contributions of the paper are not clear. The authors do not introduce the related work in detail, and it is not clear how the proposed method differs from the existing methods. How is the InterCC related other amortized clustering methods? Is the paper the first one that makes use of optimal transport for solving a clustering problem? \n\nThe authors might need to briefly introduce an optimal transport plan, entropic regularized Kantorovich relaxation, and how these concepts are related to a clustering problem. The assumption behind the proposed clustering method is also not apparent. Why is the relationship between the samples and reference distribution a proper way to define clusters?\n\nI am not sure whether the proposed optimization procedure converges. It looks like in Equation (11), $b_{i,j}^{(k)}$ and $C_k$ are dependent. Therefore, dividing the optimization process into IntraCM and InterCC might not guarantee the convergence of the algorithm. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_zZnrz7wEas", "reviewer": "Reviewer_9BaR", "summary": "The paper proposes an amortised clustering process model, where the clustering prior is learnt in a meta-learning setting. \n\nInference is performed cluster-wise sequentially, involving inter- and intra-cluster similarities. To avoid calculations involving distances between all pairs of clusters, the authors use a linear optimal transport strategy. \n\nThe method is demonstrated on synthetic gaussian mixture models as well as meta-learnt clustering in MNIST, Tiny-ImageNet and CIFAR-10. ", "questions": "* In section 4.1, I am confused by the statement \"In this case, the training set contains 200 samples belonging to 4 clusters\". I thought that amortised clustering methods would be trained on multiple datasets by repeated sampling from the CRP prior, with a variable number of clusters. Could you please clarify what is this \"training set\" refers to?\n\n* Are results in Tables 1 and 2 based on a single test set case, or averaged over multiple (potentially varying-sized) test sets? For example, for MNIST visualisation, a subset is used, but what about the results in Table 2? \n\nMinor: \n\n* The results show that the proposed method outperforms other amortised methods at test time. It could be interesting to see whether this is the case throughout training (e.g. how the training/test curves look like as a function of iterations)?\n\n* Right now, experiments are only presented as summary statistics. While this is common, it could be more insightful to see a qualitative toy example, e.g. some visualisation where the proposed OT-based methodology behaves differently to an existing model (say NCP). ", "limitations": "No concerns", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well-written and quite clear, though some parts (sections 3.2.1 and 3.2.2) are quite dense with equations. Minor suggestion: For easier readability, it might make sense to merge Figures 1 and 2. \n\nThe idea to make use of optimal transport within the inference scheme is interesting and novel, and the authors demonstrate that it's beneficial over the standard EM algorithm. I also liked how the connection between their OT-based approach and ISAB was outlined in the appendix. \n\nThe authors emphasise that their approach is sequential per cluster and not per data point. While true, it seems to me that a clusterwise approach has been previously presented in [Pakman et al, 2020]?\n\nI believe the significance of this type of work will remain relatively limited (In fact, I haven't seen _any_ of the existing work in amortised clustering processes being used for any non-toy tasks yet. On the other hand, one could argue that developing better models/inference algorithms can have the potential to change it -- in that case, ideally, it would be desirable to see convincing qualitative improvements over previous results, or a non-toy application example). \n\nI have some questions about the experiments, I have outlined them below. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "YXw6FaDc6EJ", "reviewer": "Reviewer_XhPA", "summary": "This paper proposes a novel amortized clustering framework coupled with two key strategies named Intra-Cluster Mixing and Inter-Cluster Coupling. Intra-Cluster Mixing utilizes optimal transport to learn the plan for distributing the mass of data to differential reference vectors. Inter-Cluster then resorts to this plan to form the next cluster and update existing clusters. According to the results on both synthetic and real-world datasets, the proposed method enjoys superior performance and computational efficiency.", "questions": "The settings of prior distribution are directly given in experiments, e.g., $l_{1,K}\\\\sim U(0,9)$ in line 302. I wonder whether the performance is sensitive to the choice of prior.", "limitations": "Limitations of this work are presented in the conclusion. This work is a technical clustering approach and has few social impacts.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe proposed method is novel and practical. In contrast to previous point-wise amortized clustering methods, the proposed framework directly models the cluster generative process. Introducing optimal transport in the E step to optimize the ELBO of mixing likelihood is also a highlight. Besides, this paper is well-organized and easy to follow. The proposed method is technically sound.\n\nWeaknesses: The explanation of Figure 2 seems too concise. It would be better if more details are provided to make the proposed learning procedure more understandable. Besides, the difference between the proposed method and the literature [23] is not clear. The authors are encouraged to provide more details on the differences.  \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept