Global-Local Dirichlet Processes for Clustering Grouped Data in the Presence of Group-Specific Idiosyncratic Variables
Authors: Arhit Chakrabarti
Yang Ni
Debdeep Pati
Bani Mallick
Venue: ICML 2025 poster (2025)
Keywords: Bayesian nonparametrics
clustering
group-specific variables
approximation error
variational Bayes
Primary area: general_machine_learning->clustering

Abstract: We consider the problem of clustering grouped data for which the observations may include group-specific variables in addition to the variables that are shared across groups. This type of data is quite common; for example, in cancer genomic studies, molecular information is available for all cancers whereas cancer-specific clinical information may only be available for certain cancers. Existing grouped clustering methods only consider the shared variables but ignore valuable information from the group-specific variables. To allow for these group-specific variables to aid in the clustering, we propose a novel Bayesian nonparametric approach, termed global-local (GLocal) Dirichlet process, that models the "global-local" structure of the observations across groups. We characterize the GLocal Dirichlet process using the stick-breaking representation and the representation as a limit of a finite mixture model. We theoretically quantify the approximation errors of the truncated prior, the corresponding finite mixture model, and the associated posterior distribution. We develop a fast variational Bayes algorithm for scalable posterior inference, which we illustrate with extensive simulations and a TCGA pan-gastrointestinal cancer dataset.

Global-Local Dirichlet Processes for Clustering Grouped Data in the Presence of Group-Specific Idiosyncratic Variables
Arhit Chakrabarti
Yang Ni
Debdeep Pati? Bani K. Mallick
Abstract
cancers, enabling systematic pan-cancer classification. While molecular data (e.g., mRNA expression) are shared across tumors, clinical variables may be cancer-specific (c.g., prostate-specific antigen for prostate cancer). These cancer-specific clinical variables may provide valuable infor- mation in clustering. Moreover, it is of scientific interest to investigate if patients with different clinical characteristics show differential gene expression patterns. Thus, while it is desirable to utilize both molecular and clinical information to identify pan-cancer subpopulations, their varying avail- ability across cancers makes it a challenging problem. This paper introduces a novel Bayesian nonparametric method for clustering such grouped data.
We consider the problem of clustering grouped data for which the observations may include group-specific variables in addition to the vari- ables that are shared across groups. This type of data is quite common; for example, in cancer ge- nomic studies, molecular information available for all cancers whereas cancer-specife clinical information may only be available for certain can- cers. Existing grouped clustering methods only consider the shared variables but ignore valuable information from the group-specific variables. To allow for these group-specific variables to aid in the clustering, we propose a novel Bayesian non- parametric approach, termed global- local (GLo- cal) Dirichlet process, that models the "global- local" structure of the observations across groups. We characterize the GI Goca Dirichlet process us- ing the biekkbbeakiin representation and the rep- resentation as a limit of a finite mixture model. We theoretically quantify the approximation er- rors of the truncated prior, the corresponding finite mixture model, and the associated posterior distri- bution. We develop fast variational Bayes algo- rithm for scalable posterior inference, which we illustrate with extensive simulations and TCGA pan-gastrointessinal cancer dataset.
The Dirichlet process (DP, Ferguson, 1973) is at the core of numerous model-based Bayesian nonparametric clustering methods (Antoniak, 1974; Escobar & West, 1995; Mallick & Walker, 1997; Hjort et al., 2010; Müller et al., 2015). One of the advantages of DP mixture models (Lo, 1984; Escobar & West, 1995; Maceachern Müller, 1998) is its ability to perform clustering without having fix the number of clusters a priori.
When analyzing grouped data (e.g.. tumor tissues of differ- ent origins in our application), one could naively apply a separate DP mixture model to each group, treating them in- dependently. However, is often desirable to identify group- specific clusters while allowing the groups to be linked so that clusters are comparable across groups The hierarchical Dirichlet process (HDP, Teh al., 2006) is a remarkable contribution in this direction. The HDP formulation relies on modeling groups of observations using distinct DPs with a common base measure, which, in turn, is itself a realization from another DP. Since the draws from this DP are almost surely discrete, all group-specific distributions share the same set of atoms. Another classic group clustering method is the nested Dirichlet process (nested DP, Rodríguez et al., 2008), which focuses on simultancously clustering groups as well as observations within each group cluster. However, the nested DP is known to suffer from degeneracy property (Camerlenghi et al., 2019) two distributions sharing even one atom in their support are automatically assigned to the same cluster.
1. Introduction
This article considers the clustering of grouped data that includes both shared variables across groups and group- specific idiosyncratic variables, as often seen in practice. For example, large-scale studies like The Cancer Genome Atlas (TCGA) provide molecular and clinical profiles across
'Department of Statistics, Texas A&M University, College Station, TX, USA 'Department of Statistics, University of Wis- consin, Madison, WI, USA. Correspondence to: Arhit Chakrabarti Carhit.chakrabarti @ stat tetatttam..duss
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Both the HDP and the nested DP fall under the general

---

Global-Local Dirichlet Process
framework of dependent DP (MacEachern, 1999; 2000).
hierarchical Dirichlet process mixture model (vHDPMM) for modeling grouped data that includes both shared and group-specific variables. Although the modeling motivation behind the vHDPMM is closely related to that of our pro- posed GL DP, the two approaches differ fundamentally in their underlying constructions. We detail these differ- ences in Section 2.2. Importantly, the models are not special cases of one another.
See Quintana al., 2022 for recent review of different
dependent DPs. Several recent works (Beraha et al., 2021;
Balocchi et al., 2022; Bi & Ji, 2023; Lijoi et al., 2023) have been proposed to take advantage of the cluster-sharing feature of the HDP and the group-clustering feature of the nested DP. In contrast to methods relying on the HDP or its variants, some other works rely on models with additive structure common atoms (Camerlenghi et al., 2019; Denti et al., 2023; D' Angelo et al., 2023; Chandra et al., 2023; D' Angelo & Denti, 2024). However, the aforementioned methods assume that the observations across the groups are measured on the same set of variables (with possible missing values for some variables within a group). In this paper, we develop a grouped clustering framework that explicitly accounts for group-specific variables.
First, we propose a general Bayesian nonpara- metric approach, GLocal DP, to incorporate group-specific local variables for clustering of grouped data. Second, we provide two characterizations of GLocal DP, each providing a different perspective. Third, we provide some theoretical results relating to the use of finite truncation of the Glooc DP in posterior inference. Fourth, we develop an efficient variational Bayes algorithm for scalable inference. Finally, we demonstrate our model through experiments on synthetic data as well as a real TCGA pan-gas gastroonteestiiall cancer dataset. Both simulations and real data analysis demonstrate excellent performance of our model in identifying clusters of observations shared across groups. Furthermore, our method highlights the importance of incorporating group- specific variables in refining the shared clusters into smaller subclusters through the local variables. All codes used for simulations and real data analysis, as well as the datasets themselves, are available here.
Summary:
Let Zz represent observation i from group j. They are assumed to be partially exchangeable (de Finetti, 1938), entailing that observations are exchangeable within but not across groups. Partition Zy into (a//, zff), where oft in- cludes variables shared across groups (e.g., age, sex, gene expression) and a/, includes group-specific variables (e.g.. prostate-specific antigen for prostate cancer). Unlike exist- ing group clustering methods, which consider common set of variables across groups, we distinguish between global (af)) and local (a:))) variables. We propose a Bayesian nonparametric approach to cluster observations while incor- porating this "global-local" structure.
2. Model
To model both global and local variables, we let the group- specific random measure Gj be supported on shared sub- space across groups and an idiosyneratic subspace unique to each group. Specifically, we assume that conditionally on a and V. Gj ~ DP (ax, Uy @ V). where U, is group-specific base measure, V is a common base measure, and @ is the measure product. To share clustering information across groups, we assume V is also disttriri.. This model, termed the global-local (GLocal) DP, allows Gj to share atoms in the common subspace across groups, enabling global clustering. As we will explain later, the idiosyn- cratic base measure Uy refines the global clusters into local clusters using the local variables.
Because of space limit, we provide an overview of the DP mixture model for single population/group and the HDP mixture model for multiple groups in the Appendix Sec- tion A. When data contain varying sets of variables across groups, the HDP prior (25) is not appropriate (e.g.. Gj does not have the correct support). We address this challenge of clustering grouped data with varying variable sets by proposing a joint distribution for Gj that incorporates both local and global variables.
Recall that æiji ( (aft, m)) denotes the ith observation from the group j. We assume that each observation is drawn independently from a mixture model with factor 0):- Sim- ilar to the observations, factor 0,: can be partitioned into local and global factors, 0z = (0% OC) By later con- struction, there is positive prior probability that the global factors are equal across groups (e.g., OG: = ec v thereby inducing the sharing of global clusters. Furthermore, local factors (0%) can modify the global clusters and may refine them into smaller local clusters.
GLocal DP generalizes HDP by handling group-specific variables. Unlike HDP, the GLocal DP does not assume group exchangeability. as Gj has group-specific base mea- sure. Although HDP can still be generalized to avoid the exchangeability of the groups by introducing group-specific concentration hyperparameters, the variables share the same support and, thus, HDP cannot be used to cluster observa- tions with varying variable sets across groups. Addressing this limitation and enabling the clustering of such data con- stitute the main motivation of the proposed GLocal DP.
Let F (zx 05.) be the distribution of æj conditional on factor , For simplicity, we assume that this distribution can be expressed in factorized form:
Recently, Dinari & Freifeld (2020) proposed the versatile
F(a, 10,) = (a 0%) F2 0%)
(1)
2

---

Global-Local Dirichlet Process
0f) represents the conditional distribu- tion of local variables ttj given local factors 0% while F | (tt represents the conditional distribution of global variables S given global factors 0% This implies that oj and x); are conditionally independent, although they are not independent marginally. If additional depen- dency between of and at is desired, Fi (3rf, 0%.) in (1) could be replaced with (ar) jas 0%)): however, this direction is not pursued in this paper. Let G, denote the group-specifie prior distribution for factors 0z. We assume that the factors conditionally independent given Gj. re- sulting in the following probability model:
Here, Fi (ark
man, 1994),
= Brdbr
(5)
where 3= (3k) ~ GEM(7) and Φx #d of B3. Here GEM stands for Griffiths, Engen and McCloskey distribution (Pitman, 2002). Furthermore, as each Gj is representation
H independent
distributed as DP, similar gives,
G3
Tz
(6)
where n (T ~ (E and dee V and U, @ V independent of gz. Since each factor Oz: is distributed according to Gj. it takes on the value Bhe (vfee (i))
0₁: =00 ji |G,~ 2- G2
(2)
Consider Ay) as the measurable space associated with the local factors specific to group j, and (S2, B) as the mea- surable space corresponding to the shared global factors across all groups. The GLocal DP defines a collection of random probability measures Gj. one for each group, on the product space (0, x 12, A, @ B).
with probability Tjt, where wh tid U, and oe IV Hid V. Because V has support at the points Φ (o)) the marginal distribution of each Gy with vif marginalized out also has support at these points through 4/ That is,
(199)
20 are necessarily the same as (dx) Indeed, V.y (=1
Gy a ~ DP(a, V).
(3)
takes on the value Φu with probability 3x. This sharing of global factors across the groups facilitates the sharing of clustering of the global variables. To make the clustering as- pect of our model explicit, we introduce the latent variables Lzz and kitt, where
where a is the positive concentration parameter. The base measure Uy @ V is random product probability measure of the local measure U, and the global measure V defined the product space (θ, x $2, Aj & B). In other words, Uj is defined on (9). Ay) and V is defined on (22, B). To allow for the sharing of global factors across groups, we further assume,
Lfs T ind T
kya | [ ind
B,
(8)
V Vi ~ DP(,, H).
(4)
such that, conditional on the latent indicators ty and (kyz) we have azj ~ F (arf. v (e &k ). We refer to the latent indicator kijty, as the global-level clus- ter label it indicates the shared clustering across groups. For example, if kjye; ky the i-th observation from group j and the i'-th observation from group j' belong to the same global cluster. Similarly, we refer to the latent variable lyz as the local-level cluster label as it identifies the finer sub-clusters within each group. Specifically, for two observations and i', if ty; + typ, then the local variable(s) in group j refines the corresponding global clusters kyty and kitter into two distinet sub-clusters. The dissimilarity in the local variable(s) between observations and i' drives this refinement, providing insight into how the local variables influence the clustering of global variables. With these two sets of latent indicators, we obtain an equivalent representa- tion of the GLocal DP mixture via the following conditional distributions:
where y and H are the concentration parameter and base probability measure, respectively. Equations (1) and (2) along with the prior specifications given in (3) and (4) com- plete the specification of the proposed GLocal DP mixture model. The GLocal DP reduces to the HDP when group- specific local variables (and consequently local factors) are absent across all groups. However, when local variables are present, they play a significant role in clustering grouped data. In addition to defining group-specific local clusters, the local variables can also influence the clustering of global variables across populations, as discussed the end of Sec- tion 2. This makes our method different from the HDP, even at the global level. Furthermore, following Ascolani et al. 2022, we assume non- non- informative gamma priors on the concentration parameters to avoid inconsistencies in the estimation of the number of clusters in DP mixture models (Miller& & Harrison, 2013; Yang et al., 2020).
2.1. Representations
B ~ GEM(?),
ligs ~ 3, 1/t ~ n. 1 ~ Up,
; GEM
1. THE BICK-BREAKING REPRESENTATION
Φ H.
Since the global measure V is distributed as a DP, it can be expressed using a breckbbe representation (Sethura-
Bj ~ Fi (a I 1 (te (a Onangg)
(9)
3

---

Global-Local Dirichlet Process
We remark that our clusters have hierarchical structure where the local-level clusters (given by tja) are nested within the global-level clusters (corresponding to k This hi- erarchical nature of our clusters indicates that the local variables help refine the global clusters. In our motivat- ing pan-cancer application, this plays a pivotal role in the finer understanding of molecular subpopulations modified by cancer-speeific clinical variables. The Figure in the Appendix Section B shows the graphical model representa- tion of the GLocal DP mixture model. Clearly, given the data {arfe, xf)),, the marginalized global-level assignment of observation i in group j. denoted kyt, is influenced by the corresponding local variables z/;;- Therefore, the local variables can impact the clustering at the global level.
some base measure H, and the mixing proportions T GEM(a). Introducing latent allocation variables, 2j (2i1)..., Zjng), with žÿ² = k if and only if a is drawn from global component ks equivalently, (11) is written as,
2jz fd CaI(T)))
i= ,,..,.
B2
(12)
p pl z3)
II Sa 0g 0zg).
The global cluster k is defined as Ck (x (, f=(1....)),, (=(.....) and let K be latent random variable denoting the number of global clusters. Furthermore, for each j = l....,J J and each k = I..... .. K, sf (ass) is defined as the collection of local features having the global features in global cluster k. Consequently, each sf is modeled with an infinite mixture model as,
2.1.2. THE INFINITE LIMIT OF FINITE MIXTURE MODELS
Alternaively to the stick-breaking representation, the GL.o cal DP mixture model in (9) can be derived as the infinite limit of a finite mixture model. Specifically, consider the following finite mixture model,
P(53105.T))
II E:ay
Σ
J fa Ozw):
(13)
B ~ T ~ Dir(a/T, a/T))
kst ~ B, tjj ~ J 1/ ~ Us. 30
k = 1, SJ is group- where Tfu > 0 for all k: Ew Tjw specific pdf pmf parametrized by Of nz GEM(1).
@x H.
T3 ~ (a 1e (iseey):
and 0%... ind L.j Equivalently, using hidden local clusters, (13) is defined as.
(10)
with K S T, where B3 is the global mixing proportion vector, T is the group-specific mixing proportion vector, K is the number of global mixture components, and T is the number of local mixture components. As K - the model converges to the infinite limit, which is precisely the proposed GLocal DP mixture model as shown in Appendix Section C.
I idd Bjs ~ Cat(T))) Vi s.t. = p( 10 2/)
fa 0
(14)
=
In summary, the vHDPMM defined hierarchically by first modeling the shared global variables and then conditionally modeling the local variables conditional on the global clus- ters. Contrarily, our proposed GLocal DP mixture model is defined jointly as in (9). Using our formulation, for two dis- tinct observations i and i' in the same group j. if tys = tya- then automatically, they share the same global clusters i.e.. kijt, = kj In other words, if the i-th and the i' ob- servation from group share the same local cluster, then they also belong to the same global cluster. However, this is not the case for the vHDPMM of Dinari & Freifeld, 2020, where the local clusters for observations i and i' are defined conditional on their global clusters. In other words, for any group j. if i E of and i' E s where & + k', then the two observations i and i' cannot have the same global cluster, even if they share the same local feature. Furthermore, our model exactly reduces to the HDP in the absence of local variables for all the groups However, the vHDPMM, even in the absence of local variables for all the groups, is not exactly the HDP mixture model. Additionally, the posterior inference procedures for the GLocal DP and the vHDPMM are distinct, as discussed the end of Section 4.
2.2. Comparison with Versatile Hierarchical Dirichlet Process
The vHDPMM (Dinari & Freifeld, 2020) was introduced to model grouped data comprising both shared and group- specific variables/features, aligning in motivation with our proposed GLocal DP. Despite this similarity in modeling objectives, there are crucial differences in the modeling formulations of the two approaches. Notably, the Gooal DP mixture model and the vHDPMM are not special cases of one another. To model the global (shared) variables Ej Dinari Freifeld (2020) employ a HDP-rype mixture model of the form:
G
8
p(( ( ((0)))100 O.T,)
3 flzfi 0x).
(11)
where Tyke > 0 for all k. Ek= TyA = 1, f denotes a group- independent probability density function (pdf) or probability ad mass function (pmf) parameterized by 0z, with 0g H for
4

---

Global-Local Dirichlet Process
3. Truncation Approximation Bounds
I 2 6,
R2 II
P(ONG))) pr pk (dV))
In Section 2.1.2, we presented the finite mixture model representation of the GLocal DP. derived from the finite truncation of the GLocal DP prior. The finite truncation is critical in many Bayesian nonparametric posterior inference algorithms including ours. Therefore, it is important to evaluate the error arising from the truncated GLocal DP prior, the corresponding GLocal DP mixture model, and the resulting posterior distribution.
(20)
as the prior distribution of the parameters O under the GLo- cal DP and its corresponding truncated version after integrat- ing out the random distributions. Here P(0,s G2) denotes the prior distribution of the parameters 2 given the random measure Gj. Furthermore, let m (ar) and m (x) de- note the marginal distribution of the data derived from these priors. Then we have the following result.
We recall that G,la, V ~ DP Uy & V). where Vhy DP(7, H). We hierarchically define the truncated versions of Gy as follows,
Proposition 3.1. Let p (0) and pT, K (0) denote the prior distribution of the parameters under the GLocal DP prior and its corresponding truncated version with the ran- (15) dom measures integrated out. Furthermore, let m (z) and m²'.K (x) denote the marginal distribution of the data x. derived from
K
Vk_
kk 8
80m
from these priors. Then,
rid ~ where dh
H, k = I,..., K. and
JxN | (x)) m (x000(())))) dx S
3K {
Bk
if K - 1, if K,
(16)
1- --EE Br
K-1
JNN p (0) -0)- poc.cc (0) d0 S T. (a,7),
and
where
T
G.
Tt
T.K
8ayee
(17)
T-1
N
1=1
T.K (ax 7) =4
1 a
where Uiye
ind ~
U, & V t = 1..... T, and
K-1
7
T.K J)
Jt
(18)
1 1- - Rtt
T-1
if
N = + n., EN = ( x Q) and N
Here T, K > 0 define the truncation levels for the different random probability measures.
denotes the sample spuce of observations x.
Note that the bounds approach zero in the limit and hence, the truncated prior and the prior predictive distribution ( marginal data distribution) converge in total variation (and therefore in distribution) to the GLocal DP. Furthermore, the approximation errors decay exponentially in both 7 and K. Consequently, we have the following result relating to the posterior distribution of the parameters f under the truncated Gl ooal DP prior.
Consider J groups, each of them containing nj obser- vations, j = I,...,J. Denote by (a/, x = æ; = (EF1...., ) the collection of all observations from the j-th group arising from the mixture model Zzi 03: F(-10j)) with 05:|Cr ~ G5 where the G5 's are generated according to the proposed GI Gooa DP. Let f-(00;;) be the density of F(.0),) 0xx) with respect to some dominating mea- sure. We assume that 8y: E (θ, x Q2). where (o) x 22) is a Polish space equipped with its corresponding Borel o o-field Aj @ B. Finally, we denote by x (at,,.... )) the vector containing the observations from all J groups Define
Proposition 3.2. The posterior distribution of the parame- ters O under the GLocal DP prior and its truncated version,
[x) po0.00 (0)
n 00,00 (0)m)
m00,0000 (x)
p (0)
fal10) (i)
π T.K (0)x)
P(O p()))))))) pro(dcsii) (aV)
mT.((x) (x)
satisfies
(19)
/
7 T.K (T.K(0))=-) - (()z)) m 2000 (x) de dx
and
JxN 2N
pI.K
= =0 (((
5

---

Global-Local Dirichlet Process
Thus, Proposition 3.2 tells that the posterior distribution for 0 under the truncated GLocal DP is exponentially accu- rate when integrated with respect to the marginal density of the data m (x))) under the original GLocal DP.
v @ (t, k; u, v (ug, ARJk=1 (lpye: 1 1 and the set of variational parameters is X (p.5, b, " 51,,,,,, m, m,t, D, {mj)}= (t) (Dyl/=))
Finally, the variational parameters X that maximize the ELBO are found by the coordinate ascent variational inference (CAVI see, for example, Bishop, 2006) algorithm; see the Algorithm 2 in Appendix F, where we also provide additional details on the evaluation of the ELBO.
4. Variational Posterior Inference
The standard posterior inference approach is Markov chain Monte Carlo (MCMC). However, it is well-known that MCMC methods have limited scalability when dealing with large datasets and/or in high-dimensional settings. We in- stead develop a variational posterior inference (VI) algo- rithm for scalable computation, which aims to find, among a set of simple distributions (called variational distributions), the one that minimizes the gence from the posterior distribution, which is equivalent to maximizing the evidence lower bound (ELBO). We adopt mean-field approach, which assumes the variational distribu- tions are factorized. Furthermore, we assume multivariate Gaussian likelihood for both global and local variables, al- though, this approach can be adapted in a straightforward manner whenever the data distribution is a member of the exponential family, discussed in Blei & Jordan, 2006. We refer the reader to Appendix Section F for more details on the assumed form of the likelihood, prior distributions, and an alternative finite mixture model representation of the GLocal DP that simplifies the VI algorithm.
As noted at the end of Section 2.2, the posterior inference strategy for our proposed GLocal DP differs fundamentally from that employed in the vHDPMM of Dinari & Freifeld (2020). Specifically, their approach utilizes a split-merge MCMC sampler that jointly accounts for both global and local variables. In contrast, we adopt a scalable variational inference framework, which is more computationally effi- cient and better suited for large- scale applications.
(KL) diver-
5. Experiments
We conduct experiments to demonstrate the usefulness of the proposed GLocal DP on both synthetic data as well as a publicly available pan-cancer genomics data.
5.1. Simulations
Throughout the simulations, we considered three groups or populations. We assumed that there were three shared global variables across the populations and there were two, three, and four local variables for populations 1, 2, and 3, respectively. We varied the degree of separation in the local variables for the three populations. The detailed simulation strategy is presented in Appendix Section G.4.
Following Blei & Jordan (2006), we use a truncated varia- tional distribution to deal with the nonparametric mixture, where the truncation levels are denoted by T and K. Specif- ically, we assume,
J m2 qlt, k u, U. d w 7;
J T
We assessed the clustering accuracy by adjusted Rand in- dex (ARI, Hubert & Arabie, 1985). We compared GLo- cal DP with HDP in terms of global-level clustering and with Gaussian mixture model (GMM) applied to each group separately in terms of local-level clustering. HDP used global variables only whereas GLocal DP and GMM used both global and local variables. All simulations were replicated 50 times. We ran the HDP MCMC sampler for 50,000 iterations. The first half of the iterations were dis- carded as burn-in, and posterior samples were retained at every 25th iteration after burn-in. For the proposed VI, we considered the difference in ELBO in successive iterations,
(0 (5a)7
IIII
(kks
()) x
j=1==1
T-1
K-1
II
g qugrr ages byt)
II
(vn ; ag. by) x
j=
K
II q(MR, Au; my, de. cx. k=1
Cr, D&) ((1711122
J.
(9pa A3s mj, Ag,, Cjc, Dzt) q(o; 81, 82), j=1==1
IIII
A 1,t) 10-5 as stopping rule to define the conver- gence of the algorithm. Although the discrepancy between the variational distribution and the target posterior is re- duced at each iteration, there is no guarantee that the CAVI algorithm will converge to a global optimum; rather it will likely obtain a local solution depending on the initialization. Hence, we executed 20 distinct runs of the algorithm with different starting points, keeping the one with the highest
where at (()))))) and g(kzes {p are multino- mial distributions; q(vxi āk, bx) and g(urri ajt, b,t) are beta distributions, and they are such that q(ux 1) = and g(1: = 0) = 1 for > K and similarly, q(uT = 1) = and q(uan = 0) = 1 for h > T; q(a; 81, 92) and ((1))))72) are gamma distributions; q14,, As: my, A4. Cr. D₂) and q(Pa, Ase; m2, Aye. Cjt, Dye) are normal- -ishart distribu- tions. Under this representation, the set of latent variables is
6

---

Global-Local Dirichlet Process
1.00
with the increasing separation in the local variables. More- over, the local level, the GLocal DP clustering accuracy is higher than the GMM. In the Appendix Section G.4 we present additional simulations comparing the GLocal DP with HDP and GMM with different dimensions of global variables and varying sample sizes. In all cases, GLocal DP outperforms both the competing models in terms of clustering accuracy.
0.75
0.50
0.25
0.00
1.00
0.75
Method
0.50
Goccl DP Fi
0.25
HDP
0.00
1.00
Additionally, have performed simulations comparing the clustering accuracy of the GLocal DP using MCMC vs VI. The results are presented in Appendix Section G.1. In summary, the clustering performance the two algorithms is comparable. However, VI offers significant advantages in computational efficiency and resource utilization.
0.75
0.50
0.25
0.00
Population
Population 2
Population 3
(a) Global-Ieve clustering.
Given the shared modeling motivation between the vHDPMM and our GLocal DP, we also performed se- ries of simulation experiments to compare the clustering performance of the two models under various scenarios, with detailed simulation strategies and results provided in Appendix Section G.6. In summary, both models perform comparably well in scenarios with well-separated clusters, regardless of the data-generating mechanism. However, un- der less separated conditions, the GLocal DP outperforms the vHDPMM even when the latter is the true generative model, highlighting the advantages of the GLocal DP's joint modeling framework over the hierarchical modeling approach of the vHDPMM. These results underscore the greater flexibility of the GLocal DP in capturing complex global and local clustering patterns across diverse settings.
1.00
0.75
0.50
0.25
1.00
0.75
Method Goccll DP Ti
0.50
0.25
GWM
0.00
1.00
0.75
0.50
0,25
0.00
Population
Population 2
Population
(b) Local-level clustering.
5.2. Real Data
Figure 1. Comaprison of clustering accuracy Gl.ocal DP (a) HDP at the global-level and (b) GMM the local-level with varying separation the local variables.
In this section, we showcase the usefulness of the proposed GLocal DP by analyzing pan-cancer genomics dataset. In- tegrated clustering analyses across cancers can objectively identify cancer subpopulations beyond the tumor site of ori- gin, enhancing our understanding of both intra-tumor and inter-tumor heterogeneity and potentially repurposing exist- ing cancer treatments across tumor types (Schein, 2021; Ro- drigues et al., 2022). In databases like TCGA, genomic data are often accompanied by clinical data, providing largely orthogonal information regarding tumor heterogeneity, and some clinical data may be cancer-specific. Existing meth- ods for clustering grouped data are limited to using a com- mon set of variables, necessitating the exclusion of critical cancer-specific clinical data. In this application, we aim to identify pan-cancer subpopulations using both shared and cancer-specific data through GLocal DP.
ELBO to draw the inference. Furthermore, chose the truncation levels, K T 30.
For the HDP MCMC sampler, we estimated the clusters by minimizing the variation of information loss (Wade & Ghahramani, 2018). For the proposed VI algorithm, it re- turns the optimized variational parameters corresponding to the cluster assignment probabilities, i.e., Azee= = q" (kja k) and E = q (lyr = t) where (-) denotes the varia- tional probability under the optimized variational parame- ters. Hence, in this case, the clusters were estimated
kyf = arg max Prek and k= B=.l...,K
Zgs = arg max Eiee t=I.....
We analyzed four gastrointestinal (GI) tract cancers: esophageal, stomach, colon, and rectal. Esophageal and stomach cancers, categorized as upper GI tract cancers, originate in the food pipe and stomach lining, respectively. Colon and rectal cancers, collectively termed colorectal can-
for j = 1, J. 1,... T. and 1,...,
Clearly, Figure shows that the clustering performance of the proposed GLocal DP is better than HDP. Furthermore, the clustering performance of our method clearly improves

---

Global-Local Dirichlet Process
cer (Paschke et al., 2018), belong the lower GI tract and share many features (Libutti et al., 201 8a;b). Our study ex- plores tumor heterogeneity within and across these cancers, using gene expression and clinical data from the TCGA database (Goldman et al., 2020). The dataset includes log- transformed gene expression for 60,483 genes across 173, 407, 512, and 177 patients with esophageal, stomach, colon, and rectal cancers, respectively. The selection of clinical variables to include in our analysis is explained in the fol- lowing. First, smoking has been identified as a major risk factor for esophageal cancer (Fan et al., 2008). Second, the number of positive lymph nodes serve as an indicator for the degree of tumor spread in stomach cancer (Wu et al., 1996). Third, CEA is an important prognostic marker for moni- toring tumor progression in colorectal cancer. However, CEA is not collected for esophageal and stomach cancers. Finally, recent studies have shown that several common cancers including colon cancer have been linked to obesity (Pati et al., 2023). According to Frezza et al., 2006, measur- ing BMI is crucial for assessing the obesity-related risk of developing colon cancer. In conclusion, such scientifically relevant aspects led us to consider the number of cigarettes smoked per day as local variable for esophageal cancer, the number of positive lymph nodes for stomach cancer, the pre-operative and pre-treatment CEA as the local variable for both colon and rectal cancers, and BMI as an additional variable specific to colon cancer. After excluding patients with missing clinical data, the final sample sizes were 92, 363, 173, and 120 for esophageal, stomach, colon, and rectal cancers, respectively.
to each other. And they are also not similar to lower GI cancers.
Figure 2(b) shows the subclusters identified at the local level, which emerge through the refinement of global-level clus- ters due to the presence of group-specific clinical variables. Figure 3 shows how the local-level clusters are influenced by the local variables. For instance, distinct clusters are observed among esophageal cancer patients based on smok- ing history, highlighting group-specific heterogeneity. The corresponding gene expression patterns for these subgroups of patients may be investigated based on the local-level clusters. To understand if the identified cancer subpopu- lations possibly inform cancer prognosis, we plotted the Kaplan-Meier survival curves for each of the identified can- cer subpopulations in Figure 4. Among esophageal cancer patients, the survival curves reveal distinct trajectories as- sociated with smoking history, underscoring the prognostic implications of this variable. Furthermore, for stomach can- cer, the subcluster 2g (Figure 3) is characterized by patients with a very high number of positive lymph nodes. The corresponding survival curve in Figure 4 indicates that this subgroup exhibits poorer survival outcomes compared to other groups, particularly in comparison those with fewer positive lymph nodes. Clustering based on gene expression data alone cannot discern the tumor heterogeneity from the prognostic perspective.
6. Conclusion
We have introduced the GLocal DP as stochastic process for grouped random measures and as prior for group clus- tering that accommodates varying variable sets. The GLocal DP was characterized using its stick-breaking representation and as a limit of a finite mixture model, with truncation error results providing practical guidelines for truncation level selection. We have developed a novel variational inference algorithm for scalable inference and have showcased our method through extensive simulations and application to a pan-cancer dataset integrating shared gene expression and cancer-specific clinical data. For example, our approach identifies global clusters shared across cancers and finer cancer- sancer-specific sub-clusters using local variables, which existing methods cannot achieve.
Following the common practice, performed PCA on the combined gene expression data from the four cancers and retained the top ten principal components (PC) as the global variables. We considered the truncation levels, K = T = 30 and ran the VI algorithm 20 times with different initializa- tions in parallel, choosing the one with the highest ELBO to draw inference. The maximum runtime over the 20 runs was less than 9 minutes on MacBook Pro with M1 chip and 16GB RAM. For visualization, we reduced the original combined gene expression data to two dimensions using the uniform manifold approximation and projection (UMAP, McInnes et al., 2018). Figure 2 shows the UMAP embed- dings colored by the estimated global- and local-level clus- ters. The global-level clusters in Figure 2(a) show that patients with colon and rectal cancers share significant simi- larities. However, some colon cancer patients exhibit slight differences in gene expression patterns (corresponds to clus- ter 26). Further investigation reveals that the colon and rectal cancer patients corresponding to cluster 12 have sim- ilar CEA levels (median values of 3.2 and 3.625 ng/mL.) while the colon cancer patients belonging to cluster 26 have higher median CEA of 4.1 ng/mL. Contrary to the lower GI tract cancers, the two upper GI cancers are not similar
There are few possible future directions. First, our model could be extended to incorporate the group-clustering fea- ture of the nested DP alongside the cluster-sharing feature of the HDP (Beraha et al., 2021; Balocchi et al., 2022; Li- joi et al., 2023), or to leverage shared-atom nested models (Denti et al., 2023; D' Angelo et al., 2023; D' D'Angelo & Denti, 2024). Second, in the presence of external predictors, our model may be extended to achieve covariate-assisted GLocal clustering in line with Ren et al., 2011; Wade et al., 2014; Rigon & Durante, 2021; Zhang et al., 2024.
8

---

Global-Local Dirichlet Process
10
12 12) 26a
26d
Cluster
121 12)
200 25g
122
12m
20c
Survivi probaniltty
Clasaers
0.50
0.25
0.00
2000
Time
(a) Colon Cancer.
UMAP
(a) Global-level clusters.
1.00
Colos Cancer
Exephogeel Concer
Rectal Carcer
Stemach Cencer
0.75
0.50
1000
2000 Time
3000
4000
123
(b) Rectal Cancer.
12e
Clister 15a 150 15c
1id
20
5
10-10
-5
10-10
10-10
10
0.75
(b) Local-level clusters.
0.50
Figure Global variables. (a) The colors indicate global-level clusters estimated from GLocal DP. (b) The colors indicate the estimated lecel-level clusters.
0.25
0.00
1200
Colon Cancer
Esophageal Cancer
Time
Classer
12e
(c) Esophageal Cancer.
13m
0.5
1.00
0.8
1003
200
300
Rectal Cancer
Stomach Cancer
0.50
Claster
0.25
0.2
1000
2000 Time
3000
4000
CC.AA Ienel
Number positive nadies
Stomach Cancer.
Figure Kernel density/scatter plot of local variables, colored by the estimated local-level clusters.
Figure 4. Survival curves according to local-level clusters for the different cancers.

---

Global-Local Dirichlet Process
Acknowledgments
doi: 10.1214/1-BBA116, URL htttttp:://dd
The authors would like to thank the Program Chairs, the Area Chair, and the three amonymous reviewers, whose feedback led to substantial improvement of the paper.
Chandra, N. K., Sarkar, A., de Groot, F., Yuan, Y., and Müller, P. Bayesian Nonparametric Common Atoms Regression for Generating Synthetic Controls in Clin- ical Trials. Journal of the American Statistical As- sociation, 1186544)2301-2333 2023, doi: 10.1080/
Ni's research is partially supported by NIH ROI GM148974 and NSF DMS-21 12943. Pati's research is partially sup- ported by NIH RO1 DE031134, NIH R21 DE031879 and NSF DMS-2413715.
URL https: : 222224455..22222233888383 2231581.
Impact statement
D' Angelo, L., Canale, A., Yu, Z., and Guindani. M.
Bayesian Nonparametric Analysis for the Detection of Spikes in Noisy Calcium Imaging Data. Biometrics, 79 (2):1370-138) 1382, 2023, doi:
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
13626. URL https: on inel braar. .or.yy...... wi ley. com/ com/doi/ /t///oi//aaa abs 10. /ttt/////..mm... 13626.
de Finetti, B. Sur la condition d' d'equivalence partielle. Actu- alités Scientifiques et Industrielles, 739:5-18, 1938.
References
Antoniak, C. E. Mixtures of Dirichlet Processes with Appli- cations to Bayesian nonparametric problems. The Annals of Statistics,, 2(6):1152-1174, 1974. ISSN 00905364. doi: https//tbo..rg/10..2/4/o//////// // //www. www.:jsttorrrttaaoa....o wtt.://taae/ee////33333
Denti, F., Camerlenghi, F., Guindani, M., and Mira, A. Common Atoms Model for the Bayesian Nonparametric Analysis of Nested Data. Journal of the American Statis- tical Association, 1 ):4055))405-41 2023. doi: 10.1080/ 01621459.2021. 1933499999 URL https: /ttps://do/ /oo.orgg////oo.ooooooo./.o/ooorr 10 01000/011214555 2021. 2931222444...222119999999999999 1933499. 37089274.
URL http:
Ascolani, F., Lijoi, A., Rebaudo, G., and Zanella, G. Clustering Consistency with Dirichlet Process Mixtures. Biometrika, 110(2):551-558 09 2022. ISSN 1464- 3510. doi: 10. 10.1009//iimmetaaacc0111.. URL https: //doi /ro//r/g////// 10.
PMID:
Dinari, O. and Freifeld, O. Scalable and Flexible Clus- tering of Grouped Data via Parallel and Distributed Sampling in Versatile Hierarchical Dirichlet Processes. Peters, J. and Sontag, D. (eds.), Proceedings of the 36th Conference on Uncertainty Artificial Intel- ligence (UAI), volume 124 of Proceedings of Machine Learning Research, pp. 231-240. PMLR, 03-06 Aug 2020. httpttss: pttpp//deceegs..///// ml mlr.press 1124//inari20.. html.
assa0011
Balocchi, C., George, E. I., and Jensen, S. T. Clustering Areal Units at Multiple Levels of Resolution to Model Crime in Philadelphia, 2022. URL
htg//bb/// 2212. 02059.
Beraha, M., Guglielmi, A., and Quintana, F, A. The Semi- Hierarchical Dirichlet Process and its Application to Clus- tering Homogeneous Distributions. Bayesian Analysis, 16(4):1187-1299, 2021. doi: https///doiorg/11.11144 21-BA1278.
D' Angelo, L. and Denti, A Finite-Infinite Shared Atoms Nested Model for the Bayesian Analysis of Large Grouped Data Sets. Bayesian Analysis, pp- 1-34, 2024. doi: 10.1214/24- URL https:/// / http://oo..o/// org/ 10. Escobar, M. D. and West, M. Bayesian Density Esti- mation and Inference Using Mixtures. Journal of the American Statistical Association, 90(430):577-588, 1995. doi: 10. 00.108000162145911995 1995144445999999.11665 10476550. URL
Bi, D. and Ji, Y. A Class of Dependent Random Distribu- tions Based on Atom Skipping, 2023.
Bishop, C. M. Pattern Recognition and Machine Learning, volume 4 of Information Science and Statistics. Springer, New York, 2006.
Blei. D. M. and Jordan, M. I. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1(1):121 - 143, 2006. doi: 10. 19.121006-BA1044 URL https: //doi. org/10 121 0/144/06600 06B4A104.
https: http:://wwww.o http///wtaaat ttndffannnnnnnae com/ do//dboiabb////////// 10. 00110000001215555
621 626.9211199.0 1995. 10465500.
Fan, Y., Yuan, J. M., Wang, R., Gao, Y. T., and Yu, M. C. Alcohol, Tobacco, and Diet in Relation to Esophageal cancer: The Shanghai Cohort Study. Nu- trition and Cancer, 60(3):354-366 363, 2008, doi: 10.1080/ 01635580701 883011.
Camerlenghi, F., Dunson, D. B., Lijoi, A., Prünster, I., and Rodríguez, A. Latent Nested Nonparametric Priors (with
Discussion). Bayesian Analysis, 14(4):1303 1356, 2019.
10

---

Global-Local Dirichlet Process
Ferguson, T. S. A Bayesian Analysis of Some Nonparamet- ric Problems. The Annals of Statistics, 1(2):209-230, 1973. doi: 00.211/aas/111634422
MacEachern, S. N. Dependent Dirichlet Processes. Tech- nical report, Department of Statistics, The Ohio State University, 2000.
URL https:
Maceachern, S. N. and Müller, P. Estimating Mixture of Dirichlet Process Models. Journal of Compu- tational and Graphical Statistics, 7(2):223-238, 1998. doi: 10. 111000000006000 199..19948.7777 URL https: : //www http::/wwwwwwww.ow/wettt/atcoaaa ine www.///o/////eooooooooc/oocc.ooe abs/ 10. 1080 001 8600 1 1988 0474772.
Frezza, E. E., Wachtel, M. S., and Chiriva-Internati, M. Influence of Obesity on the Risk of Developing Colon Cancer. Gut, 55(2):285-291, 2006. doi: 10.1136/gut. 2005.073163.
Goldman, M. J., Craft, B., et al. Visualizing and In- terpreting Cancer Genomics Data via the Xena Plat- form. Nature Biotechnology, 38(6):675-678 2020. doi: 00.10888/415877-00--5552 8. URL attps://doi. org/10. 1038 /ttp//00110///44115111 s41587 020 0000-000000------- 0546-8.
Mallick, B. K. and Walker, S. G. Combining Informa- tion from Several Experiments with Nonparameter Priors. Biometrika, 84(3):697-766, 1997. ISSN 00063444. URL http:/ //wp:///wwwwwww istor org/ sttps/aaalllbee// /at2lle223355...
Hjort, N. L.. Holmes, C., Müller, P., and Walker, S. G. Bayesian Nonparametrics, volume 28 of Cambridge Se- ries Statistical and Probabilistic Mathematics. Cam- bridge University Press, 2010.
Manning, D., Raghavan, P., and Schlitze, H. Introduction to Information Retrieval. Cambridge University Press, Cambridge, 2008
Hubert, I.. and Arabie, P. Comparing Partitions. Journal of Classification, 2(1):193-218, 1985. doi: 10.1007/ BF01908075. URL https: http::::/ddi.rrror oto. 1007/ BF01908075.
McInnes, L., Healy, J., Saul, N., and Großberger, L. UMAP: Uniform Manifold Approximation and Projection. Jour- nal of Open Source Software, 3(29):861, 2018. doi: 10.21105/jjss.00866 URL https:://do org/10. 21105/ joss. 00861.
Ishwaran, H. and Zarepour, M. Exact and Approximate Sum Representations for the Dirichlet Process. The Canadian Journal of Statistics Revue Canadienne de Statistique,
Miller, J. W. and Harrison, M. T. A Simple Example of Dirichlet Process Mixture Inconsistency for the Number of Components. In Burges, C., Bottou, L., Welling, M., Ghahramani, Z., and Weinberger, K. (eds.), Advances in Neurul Information Process- ing Systems, volume 26. Curran Associates, Inc.. 2013. URL https://proceeddngs neurips. cc cc/paper cc/paper_filestt /iaaaeerffLlesspppee//2////// TT6665504c666222227777776683 pdf.
30(2):269 283, 2002. ISSN 03195724. doi: https:// dbi.orrg/1022307733315555 URL http: //ww///wwoooooo////o jstor. htt/stable/331599
Libutti, S., Saltz, L., Willett, C., and Levine, R. Cancer of the Colon, chapter Cancer of the Colon, pp. 918-970. Wolters Kluwer Health Pharma Solutions (Europe) Ltd, 2018a. ISBN 9781496394637.
Libutti, S., Willett, C., Saltz, L., and Levine, R. Cancer of the Rectum, chapter Cancer of the Rectum. Wolters Kluwer Health Pharma Solutions (Europe) Ltd, 2018b. ISBN 978 7781496394637.
Müller, P., Quintana, F. A., Jara, A., and Hanson, T. Bayesian Nonparametric Data Analysis, volume 1. Springer, 2015.
Paschke, S., Jafarov, S., et al. Are Colon and Rectal Cancer Two Different Tumor Entities? A Proposal to Abandon the Term Colorectal Cancer. International Journal of Molecular Sciences, 19(9), 2018. ISSN 1422-0067. doi: 10.3390//jjss19992577 URL https: /www mdpi com/ hom/ 0067 /9/99999/777
Lijoi, A., Prünster, I., and Rebaudo, G. Flexi- ble Clustering via Hidden Hierarchical Dirichlet pri- ors. Scandinavian Journal of Statistics, 50(1):
213-234, 2023. doi: https://doi.or//10.11111 wiley..
12578. URL http///..../sss....eoaao 12578.
com/do /////o///bb///
Lo, A. Y. On Class of Bayesian Nonparametric Estimates:
Pati, S., Irfan, W., Jameel, A., Ahmed, S., and Shahid, R. K. Obesity and Cancer: A Current Overview of Epidemiol- ogy, Pathogenesis, Outcomes, and Management. Cancers, 15(2):485, 2023. doi: 1.399//ancees1550200
I. Density Estimates. The Anmals of Statisties, 12(1):351-
357, 1984. doi: /0.1214/ao/1117344466
URL https:
//doi. org/ /tl.o////./////////// 10.10000 1211 10w..//1/1711411..
MacEacher,, S. N. Dependent Nonparametric Processes. In ASA Proceedings of the Section on Bayesian Statisti- cal Science, Alexandria, VA, 1999. American Statistical Association.
Pitman, J. Poisson-Dirichlet and GEM Invariant Distribu- tions for Split-and-Merge Transformations of an Inter- val Partition. Combinatorics, Probability and Comput- ing, 1155):001-514 sep 2002. ISSN 0963-5483. doi:

---

Global-Local Dirichlet Process
10. 10.0017/5006334433000066611 URL https::tt//// //dii. org/10.
Wade, S., Dunson, D. B., Petrone, S., and Trippa, L. Im- proving prediction from dirichlet process mixtures via enrichment. Journal of Machine Learning Research, 15(301:04411071,, 2014. URL http: jttp://///mmr hmmp:.//mmmoorr/ orq/ papers/vil http:////.wwwweeee//////ee html.
Quintana, F. A., Müller, P., Jara, A., and MacEachem, S. N. The Dependent Dirichlet Process and Related Models. Statistical Science, 37(1):24 41, 2022. doi: 10.1214/ /ww.ooiiioooo 20-STS819. URL http:////2.rro//.2111... 20- STS819.
Wu, C. W., Hsieh, M. C.,, Lo, S. S., Tsay, S. H., Lui, W. Y., and P'eng, F. K. Relation of number of positive lymph nodes to the prognosis of patients with primary gastric adenocarcinoma. Gut, 38(4):525-527, 1996. doi: 10. L336/gut.3..4.25. URL attpp::/ gut bmj com/ content/38 /3///////44/55////
Ren, L., Du, L.,, Carin, L., and Dunson, D. Logistic Stick- Breaking Process. Journal of Machine Learning Re- search, 12(7):203-239, 2011. URL http:// jmlr. html.
org/papprs/v12//eee
Yang, C.-Y., Xia, E., Ho, N., and Jordan, M. I. Posterior Distribution for the Number of Clusters in Dirichlet Pro- cess Mixture Models, 2020. URL https: //arxva...vv azg/abs/1900 09959.
Rigon, T. and Durante, D. Tractable Bayesian density regres- sion via logit stick-breaking priors. Journal of Statistical Planming and Inference, 211:131-142, 2021. ISSN 0378- 3758. URL www. www.ecedieeee...c............... com/ scceence/articllees///////3333
Zhang, H., Wade, S., and Bochkina, N. Covariate-dependent hierarchical Dirichlet process, 2024. URL https:/ // arxiv. www.v.orrg/bbe. 2407 02676.
Rodrigues, R., Duarte, D., and Vale, N. Drug Repurpos- ing in Cancer Therapy: Influence of Patient's Genetic Background in Breast Cancer Treatment. International Journal of Molecular Sciences, 23(8):2280, Apr 2022.
doi: 00.33900jmm2308428000
Rodríguez, A., Dunson, D. B., and Gelfand, A. E. The Nested Dirichlet Process. Journal of the American Sta- tistical Association, 103/483) 2008. doi: 10. 10.11880166144500000000533 URL https: //doi. http://..1119/111111000000000000
Schein, C. H. Repurposing Approved Drugs for Cancer Therapy. British Medical Bulletin, 137(1):13-27, Mar 2021. doi: 10.1093/bmb/ldua045..
Sethuraman, J. A Constructive Definition of Dirichlet Priors. Statistica Sinica, 4(2):639-650, 1994. ISSN 10170405, 19968507. URL attp::/wwwt jstor.
oxg/stable/2430555
Strehl, A. and Ghosh, J. Cluster ensembles- knowledge reuse framework for combining multiple partitions. Jour- nal of Machine Learning Research, 3:583-617, 2002.
Teh, Y. W., Jordan, M. I., Beal, M. J., and Blei, D. M. Hierarchical Dirichlet Processes. Journal the American
Statistical Association, 1664477)):56611555) 2006. doi:
10. 10.11188006224455000000000000000 URL https: /dooi. org /10 1998 /11111///1111111 /tt.//00011110000000000000000000
Wade, S. and Ghahramani, Z. Bayesian Cluster Analysis:
Point Estimation and Credible Balls (with Discussion). Bayesian Analysis, 13(2):559 626, 2018. doi: 10.1214/ 17-BA1073. URL https: / dooi /ttp:iiooo....11//////1 17 1--BA1077
12

---

Global-Local Dirichlet Process
A. Background
In this section, we present brief overview of some preliminaries needed before introducing our model in Section the main manuscript. In particular, we provide concise introduction to infinite mixture models for single population, the DP mixture model, and for multiple exchangeable populations, the HDP mixture model.
DIRICHLET PROCESS MIXTURE MODEL
For single population, let z; denote the ith realization of random variable X. Consider the following mixture model,
02 G sid G, Jy 0, ind F(O.),
(21)
where F(0,) denotes the distribution of x, parameterized by 0,. The parameters 0,'s 's are conditionally independent given the prior distribution G. In DP mixture model, G is assigned DP prior, ~ DP Go) with concentration ag and base probability measure Go-
Sethuraman, 1994 presented the stick-breaking representation of the DP based on independent sequences of i..d. random variables (m)) and ( which is given by,
rid Beta a0),
Φ
iid
Go.
(22)
k-1
Th = T II (1-1)) 1=1
G
T
(23)
where de is a point mass at Φ and ou 's are called the atoms of G. The sequence of random weights π = (TT))== constructed from (22) and (23) satisfies Σk=1 T = 1 with probability one. The random probability measure on the set of integers is denoted by π ~ GEM(( for convenience where GEM stands for Griffiths, Engen and McCloskey (Pitman, 2002). clear from (21) and (23) that 0; takes the value Φe with probability Tg. Let z; be a categorical variable such that 2; k if 0: Φ2. An equivalent representation of Dirichlet process mixture is given by,
π GEM(a.).
π
(24)
OR Go.
24 Zis (o)) ind
F(oz.)
HIERARCHICAL DIRICHLET PROCESS MIXTURE MODEL.
Suppose observations are now organized into multiple groups. Let æys denote the observation from group j. Let F(0,;) denote the distribution of zy parameterized by fez, and let Gj denote prior distribution for 0zz. The group-specific mixture model is given by,
0;s G ~ Gj, Zzs 05s
ind F(O):).
As with the DP mixture model, when the random measures Gj is's are assigned an HDP prior,
Go DP(7,
(25)
Gj Go ~ DP( Go),
the corresponding mixture model referred to as the HDP mixture model. The global random probability measure Go distributed as a DP with concentration parameter y and base probability measure H. The group-speeific random measures Gy's are conditionally independent given Go and hence exchangeable. They distributed DP with the base measure Go and some concentration parameter agg. Because DP-distributed Go is almost surely discrete, the atoms of Gj 's are necessarily shared across groups. This leads to positive probability of shared clusters across different groups.
B. Graphical Representation of GLocal DP
In this section, we present the graphical model representation of the GLocal DP mixture model (Figure 5).
13

---

Global-Local Dirichlet Process
Uz
kj
H1
Figure 5. Graphical representation of the GLocal DP mixture model. Fach node corresponds to random variable, with shaded rectangles denoting observed variables. Rectangular plates indicate replication.
C. Proof of the Infinite Limit of Finite Mixture Model
The finite mixture model represenatation of the GLocal DP is given by,
8 Dir(y/ 7//(//
kjt B
7 (ifa/TT......TT)
tgi Tī
(C.26)
Φk H,
w U,
3n Fi (a 1 12 okse,,)
where 3 is the global vector of mixing proportions, n; is the group-specific vector of mixing proportions, K is the number of global mixture components, and 7 2 K is the number of local mixture components. Further, as K - oc, the infinite limit of this model is the proposed Glocal DP mixture model.
Proof. Consider the random probability measure
K
K
3rddes
where 3 = (zx) Dir(y/K,..,, Y/K) and Ob H. k k= l..... K independent of B3 Ishwaran & Zarepour, 2002 shows that for every measurable function integrable with respect to H, we have, as - co.
f ((0)ar<((0) B g(0)av
(C.27)
Further, for T K, define
G.KKK
14

---

Global-Local Dirichlet Process
Uy & V independent of πÿ. Let B, x C be an where 3y = (Tx))=1 ~ Dir(a/T,.... ................................ a/T) and dy = (wffe (t) sed
arbitrary measurable subset of 9 x Q. Then,
T
of (Bxx
Σ
Tye (1 (v4)) (c(
t=l
Σ
M (c))))
(C.28)
Here the indicator function La(x) 1 if A and is 0 otherwise. The second equality follows since for T oc and any fixed ei Φk. for some k 1,..., K. Since (C.28) holds for any arbitrary measurable B, x C, we have
GJ ~ DP(a, U, & V*).
(C.29)
It is clear from (C.27) and (C.29), that as K + T + oo, and the marginal distribution that the finite mixture model induces on the observations approaches the proposed GLocal DP mixture model.
D. Proof of Truncation Approximation Bounds
In this section we provide the proof of Proposition 3.1 and Proposition 3.2. Recall that the GLocal DP prior is defined as, Gyla, V DP U, & V), where Vly DP(7, H). Furthermore, define the truncated versions of Gj follows,
VK
3k
80
k=1
where d& 4...d. H, k- = I....,K. K.
Bk
if S K 1, K,
3 =
1
ΣK k=
K-1
BR
T
GTKKK
J.K 803 jit
1=1
where 1uz ind UUOVA t= 1,..., T, and
T.K J
Tjt
if -
T 1-
1 Jt
if T.
Here T, K > O define the truncation levels for the different random probability measures.
Consider J groups, each of them containing n, observations, j ===,, l, Denote by (a 20) = 0j = (Zj1...., Bjng) the collection of all observations from the j-th group arising from the mixture model Zzj 00 F((10 with 0 G. Gy ~ G, where the Gý² 's are generated according the proposed GLocal DP. Let f-|(0j; be the density of F(-10,,) with respect to some dominating measure. We assume that 0j: E (0₃ x I), where (9₃ x 2) is Polish space equipped with its corresponding Borel o-filld Aj @ B. Finally, we denote by = (æI,... x)) the vector containing the observations from all groups. Moreover, recall that
I PIONGG (pdd)) (dv and
000 (0)
j=1 O2
PT.k (0)
II P(O.IG)) PT PT
@
15

---

Global-Local Dirichlet Process
denotes the prior distribution of the parameters 0 under the GLocal DP and its corresponding truncated version after integrating the random distributions. Here p(0 G3y denotes the prior distribution of the parameters 03 given the random measure Gj. Furthermore, let m m00,00 (x) and m mf.k (æ)) denote the marginal distribution of the data æ derived from these priors. Particularly,
S s(z)0))TT.((0 do.
(x)
JEN Po000) d0 and m mx. =
Then we have the following result.
Proposition 3.1. Let (0) and PT.K (0) denote the prior distribution of the parameters 0 under the GLocal DP prior and its corresponding truncated version with the random measures integrated out. Furthermore, let m0,,s (a)) and m.. (x) denote the marginal distribution of the data E, derived from these priors. Then,
JeN. [m (a)) 111 (x)) da S / ENN
|P::K (0) p00,00 (0) do S 6 T.K (a,
(D.1)
where
T.K T.K(a, 7) =4 1 1 { (1-((11(10)
N
I+a
1
?
(D.2)
1+7
N = n1 n.y NN (0 (0, , and XN denotes the sample space of observations
Proof. Note that,
J XN
|mT:((a)) m xx (x)) dx
XN / ((:
[
da
((01)) (p (0) - peo, {0)} do dx
JNN XM
(0) ardd (Ola)
- 10...0 (0) d0 dx
XNN
JEN
f(z/))
1.. da
(0) 000.00 (0) d0
EN
| (0) poo000 (0) 0b
SN
2 sup pT. (A) po..o
A AEEN
= 2 drv drv(p"T. (PT.K poo,00),
(D.3)
where drv(PT.t ,00,000) denotes the distance in total variation between the marginalized GLocal DP prior on 0, p00,00 and its corresponding truncated version, PT.K Let 0,: (1), 0C jkyr,)) ) The sampled values of (0,) under GT and G' are identical when tjz < T and kjt,, K, for I,... nijjj J.. Therefore, have,
dry (PT.K x20 2 T.K {ty < T and kyts < K for all j and i})
= 2 GT.K (kyz K for all j and tye T for all j and i} GT.K K [tye < T for all and i}) n
=
1
E
Tt
E
Σ 3k
J
nh E )x) ))
2
1
E
(1
16

---

Global-Local Dirichlet Process
(11((ta))
N (1- K- 1+7
(1-(1->) )"
(11-
a lta (Ita))
= 2
(D.4)
where the second last inequality follows by the Jensen's inequality and the last equality follows by straightforward calculations based on the stick- breaking representation of the weights. The proof follows immediately from (D.3) and (D.4).
Next, we prove the Proposition 3.2, which states,
Proposition 3.2. The posterior distribution of the parameters 0 under the GLocal DP prior and its truncated version,
π 00,00 (0|z) f(ar P00,00 (0)
10,00000)
(D.5)
f(x (x) (0)
n T.K (0 (x))
mT.K(6 (x)
satisfies
J NN JNN
/ |T T: (0 x:) 00. m d0 de (e::K(a,
(D.6)
Proof. Write
p00,00 (0) = pT.K (poo,00 (T.K(())
Therefore we have,
f(x10)pT.KK PT,K
f(x: (l.(0))TTK((() (0)
-p. 000000d (0000000) (0) - pT. (. (0)
f(x(0)
π T. (0(x)) - (01(z))
+
-00000000 (n)
m (x)
000,00 (x))
((z)0))TTKK (0)
000.00 (x)) (2)
m..KK
f(z|0)
1... (0) pT.kk (0))
m=-K(a)) (x)
m00.0000 (x)
((: (0)
m..K (x))
f(x 0)
pp..0
pT.k (0)
(T))(x))
m0000000
m00000
and consequently,
-0000)))000000000 (0|xz) m (x) do dx
XN EN
d0 dx + I XN I f(x10) (0.,:' pT.k (0) d0 dx ((010)p1.K(0) T.TK(()) - 20.000()))) mm.00
$
XN JEN
m².k( (x)
((x10)PTK((0 (0)
d0 |m m 00,00 do + [ U ((l dx) (0) pT.. pT).(()) do
X WJN
m (x)
-Jrm Xxx |m T.K (xx m ( doc + I pox (0) - (0) d0
S <T 7) + T (a,7) from Proposition 3.1
= O (T. ())
which concludes the proof.
E. Blocked Gibbs Sampler
Based on the finite mixture model approximation of the GLocal DP in (10), with large enough truncation levels K and T, straightforward to develop an MCMC-based Metropolis-withh blobhi--blokke-Gibbl sampler for the GLocal DP. this section,
17

---

Global-Local Dirichlet Process
outline the blocked Gibbs sampling algorithm as an alternative our VI algorithm as described in the next section. Recall the finite mixture model representation of the GLoocll DP,
8 Dir(z/ K, /// K); J Dir(a/T Ox.)
Kizt 3
lji Tjj
(E.I)
0% ~ H,
J UJ
Bjj Fi Fa w Fa okgg,)
Let æ (w; to denote the observations from all J groups. Similarly, - (t;) and & (kz) denotes the collection of all local-level and global-level latent indicators respectively. The collection of all local atoms denoted by / = (1b)) with 3 = (1) t=1 denoting the local atoms of group j. Similarly, the collection global atoms are given by Φ ( Furthermore, let fi wht)) and fa Φu) be the density functions (with respect some dominating measure) corresponding to the distributions Fi vife) and Fa(. Φk),, respectively. The augmented likelihood then given by,
T
J nyy
J ny
T
K
p v. o. (TT))=
((x=))
j=
The model parameters are (t) Φ (T, (1,))=11 B, ax r.. with the joint prior distribution given by,
J T
K
K P((O))) p()))
( } p(3))) P())
(T))
IIII --
We remark that K and I are the maximal numbers of global and local clusters specified by the users. They should be large enough so that the numbers of sampled clusters are always strictly smaller than them over the course of the MCMC. Picking the maximal number of clusters in our algorithm much more straightforward than picking the exact number clusters in many existing clustering algorithms. Alternatively, the user should choose value for K and T that leads to precise approximation of the truncated GLocal DP mixture model.
For simplicity of presentation, for the global variables, assume mixture of multivariate Gaussian kernels p(- ()) N- μ. -)),, with p-dimensional mean vector and A precision matrix. Also, we assume a conjugate normal-Wishart prior distribution on the model parameters,
(μ, A) NW To, Po, Vo)).
Similarly, for the local variables in population j. we assume, Py wht = NNp (,1, A))), with μ4 a py-dimmessoonal mean vector and A; a P; X Py precision matrix. Furthermore, we assume a conjugate Normal-Wisharl prior distribution on the model parameters,
(1z, Az) NN Myo, Jo))
The blocked Gibbs sampler for the GLocal DP is given in Algorithm 1. We remark that for other choices of kernels and prior distributions, the blocked Gibbs sampling algorithm is straightforward to derive from Algorithm 1. Particularly, the likelihood functions in steps and 6 of Algorithm need to be replaced by those of the chosen kernels. Similarly, the conditional posteriors of the global atoms and local atoms in steps and would change depending the choice of the kernel and prior distributions.
Algorithm 1 Blocked Gibbs Sampler for the GLocal DP
for i = to B do Sample the parameters from their conditional posterior distributions,
1. For each j, sample ty, from a Dirichlet distribution Dir(m,1 ta/ ta/ttaTTT.... mar..aTTTaa where mye = E. 1(t = t).
18

---

Global-Local Dirichlet Process
2. Sample 3 from Dirichlet distribution Dir(d] + 7 .,.,,,.,,,,........... dx + t/K), where d₁ Σÿ= Et= ( k).
3. For each k, sample 6% from a conjugate (W te, lk. 4x) distribution with parameters
PA (To Mu + N1
da = + Nu. in = v0 + NA.
To (x uo) ( - -u)) T + Sk
$ v 0
50 No
where
J ny
ny T
N₂
ΣΣΣ
Σ
1(t tt) =)),
o No¹
ΣΣΣ Σ
[1ty t) 1(2gs = k) 2
J 2
Σ
((t)) k) (zff - xf) (aft - - x
4.
For each j and t, sample / from conjugate NW(pt Ege, Bge+ x(t distribution with parameters
As = Az1 (Ty Mgo +
NNyt Åg. T30 + Nges Dj 1yo + Nge.
v 1 + Jyn-
(3t/ 1130) (a/4 - -1;))
T + +
jr
jo
Tjo Nge
where
n2
N₂ = Σ
(tzi = t);
a/J = Not
Σ
1 (1/z = 1) 2tj
1=1
5/10 Σ (1(1))= 1) (2/1 - 2/j (arfi m31)²)
5.
For each j. i, and f, sample the local-level latent indicators from the following full conditional distribution:
Ptt = t -) x Nz (xo) Myt A (N Mkye AF
6. For each j. f, and k; sample the global-level latent indicators from the following full conditional distribution:
Pkky = k 1) x Bx
II
Na (s
7. Sample a following full conditional distribution:
[T(a)) J T (T(a/T))
p(a x
IIII i=1
o/T p(a),
je
using Metropolis Metropolis-Hastings Hastings (MH) step with a gamma proposal distribution.
8. Sample y following full conditional distribution:
T(7) (T(/KK}
K II
--)1-) x
k p(r),
R=1
using a MH step with gamma proposal distribution.
end for
19

---

Global-Local Dirichlet Process
F. Variational Inference
In this section, we outline the variational inference algorithm for the GLocal DP for the specific case of multivariate Gaussian likelihood. However, this approach can be adapted in a straightforward manner whenever the data distribution is member of the exponential family, as discussed in Blei & Jordan, 2006. As before, for the global variables, we assume mixture of multivariate Gaussian kernels p(- Φ) N μa, A A with μ p-dimensional mean vector and A a p x p precision matrix. Also, we assume a conjugate normal -ishart prior distribution on the model parameters,
(n, (Ho, To, Vo, Vo).
Similarly, for the local variables in population j. we assume, p abf)) N₂ (- (3, with μtj a py-dimensional mean vector and A a P3 Py precision matrix. Furthermore, assume a conjugate Normal-Wishhan prior distribution on the model parameters,
(Hg A5) ~ NW Moo, V jo).
First, we provide an alternative representation of the proposed Gl DP relying on the finite truncation of the stick breckbbr representation. Specifically, consider the following finite mixture model,
i.dd Up ~ Beta(1, 7), h = L,...,,KKK K - 1, UK = =1,
B1 = U1 Br = =t II (1 - vi), for ...
Let [ = (B,,... ... ,8k)
kst ~ B,
Ujt ind Beta(1, a), t= I,..., T 1, UTT 11,
Ty1 = Uji, Tit = Use II uze), fort 2,..., T
Let ,T, = (T......., TTr))
(Nu: A&) NW(HO: To, un, Vo),
(Ugz, Az) ~ NW Tjo, Kj0, J0),
Fz ~ Nos (o/k Mitza A st,e) (a Maggaz A Anryy)
(F.I) Furthermore, we assume non-informative priors on the concentration parameters, i.e., y Gamma by) and a Gamuna b,). Recall that we consider fully factorized mean field family of distributions as the variational family, Furthermore, we use truncated variational family to deal with the nonparametric mixture, where the truncation levels denoted by T and K. Specifically, we assume,
qt, kt u, v, u, 2, a,,,
IIII g j=li=1
II
II j=l t=1
II
( aet by)xx
j=1
K-1
K
x II
q uu CR. b&)
II
Aps. Ma du, ck. D&) x
k=
km
IIII g Agi mar Ass St. Dz) gla;s1, 92) g(33)1172) j=tt=l
where atsss (E 1 and akks {pra)=]) multinomial distributions: q(kk: āg by) and q(uze: Cze, bya) are beta dis- tributions, and they are such that q(ux = 1) = 1 and q(1, 0) = for > K and similarly, q(ur = 1) 1 and q(ujn = 0) = 1 for h > T; g(a;s1, and ((FiT1, r2) are gamma distributions; g(uk,, As; m& Ap, Ck. D&) and qH12,, AJri mi Aje Cjt, Djt) are normal-Wishart distributions. Under this representation, the set of latent vari- ables is θ = (t, k, u. U, AR]K (Pr A a, and the set of variational parameters is = ( Optimization is then car-
(p.g. a. b, (aa))=1.(b))1 (b,})= $1. $2, T2,72, m, t, c, D, {my))m ried out by looking for the combination of variational parameters X that maximizes the evidence lower bound (EI.O).. To this end, based on (F.1), we derive a coordinate-asea variational inference algorithm (CAVI), given in Algorithm 2.
Algorithm 2 CAVI updates for the GLocal DP
Input: ( 4-400 0. Randomly initialize (0) Define the threshold e and randomly set A >
while ((t -0--,, 1,t) > E do
Set = ++ 1; Let (t-1) (t);
Update the variational parameters according to the following CAVI steps:
20

---

Global-Local Dirichlet Process
j I,..., and .,.., 7, q" (Rze) is a K-dimensional multinomial, with q" (kyz = Pyck for ,,...,
R-1 9 ā.)
log Pjck g(ōz, bu) +
Σ
Σ Esit
(1))
jssk)
where g(x, y) 1b(x)) 10(z + v). with denoting the digamma function, 1 = ((ck - 1)/2) plgg2
log D&l. and (22) jik = -p/>x cs (z/) mx)" Dx( mm;).
2. For j 1,..., J and i 1, o (ftx) is a T-dimensional multinomial, with q (fys f) Ejet for l....,,
E, g(as,, byt) Σ glbs, a,) + 56
000 jst +
Pjck (e()) +
((2)
jikt
where (()) = Eee vi ((ex - - 1)/2) p5 log 2 + log Dyel and 50
ps/ Ayx excl - m2a)² Dyclasj -ma).
For j = I....,,J and ( ,,..., - 1. q. (uşe) is a Beta(ajr, bye) distribution with 3.
ajt 1+
Ejsr bys 81/52
Sja.
1mt+1
4.
For k = I....., K 1. (vx) is Beta(ax, bx) distribution with
T
āu 1+
Σ
Pycks
=
Σ ΣΣ Pzt.
jm11m1
(m141 1m1
5.
For k 1...., K. (Mr, Au) is a NWWMM Ck. D2) distribution with parameters
m. Ax (To + N1
ds = To + NN
C* = vo + Nk.
D Vo + N₂
NoN2 @ 1o)
T
140
+ S
where
ny
N
Esst Pjtks
# = No¹
Σ
Ezst Pitk azj
T
Eas Pyck (2)i - xx
(I 2x T
6. F 1,...,,J J and / 1, I..... T. 9" (pge, Art) is NW(ms;; D₁t) distribution with parameters
mii Xze (Tia 1a Naafe), Aga Tyo Nze,
Cjt = v50 N5t,
D j + NopN (#), - m30) (#)4 100)
+ S..
Tyo + Nze
where
ny N₁t
1
Exets
क = Nj²¹
Σ
Esit Tye
(=1
S/
Σ
Esit
( - /イ/
(nt/ Ttf1
(a) a Gamma(si, s2) distribution with parameters
T-1 Σ
+ J(T
ba
Σ
ol a.t).
[=1 1=1
21

---

Global-Local Dirichlet Process
8. (2) is Gamma(r1, 12) distribution with parameters
K
71 -= a, + + 1,
12 = by - Σ g(bu, ax).
Store the updated parameters in let (0) A; Compute A(t 1, = FLBO(A ( F.BB () end while Return X containing optimized variational parameters.
F.1. Computation of the ELBO in the variational inference approach
Here we outline the ELBO evaluation for the GLocal DP. Recall that we use the notation g(z, v) = - y). The minimization of the Kullback-Leibler divergence between the posterior and the variational distributions is equivalent to the maximization of the ELBO, expressed as
ELBO(g) = B, logp(æ, θ)] 15 [log ax (+)].
The first term, E, [log O)], can be decomposed into the following components:
1. Ellog(ax* Ac E En Eis Esse { ( + + P3 log 2n }
2. E'logp(= (uu; As K )1 E Ens Et= kk=l Ek=i Eset Puck { {eh) + plog2
3. E [logp(t u)] Σ/=1 Σ - 1i=1 Eye {g bjc) + 9(b,,,
4. E (logp((k w)] = Σj= Σ ΣK Pjtk (āz, bb) O(b,
5. E log P( K Lu Ax))]
poT
log Lo vo) + 0.5(Po - p p - 1) 1 (T +
0.5 0.5 k= ( 2n)) + ( pro/ A Tock (m mb) DD [o) where T(-) is the trace operator
and B L vo) ) is the inverse of the normalizing constant of a Wishart distribution (see, for more details, Appendix B
of Bishop, 2006).
6. E log 1-1 I (p Axtt) = T log B
L jo, U30)
+ 0.5 0, E/ (130 - Pj - 1)
0.5 Σ =1=1 Et= 9t T(V Djt)= + 0.5 1 Σ [pg log(T50/22
+
(()
jt
P/Tzo/Ajt TyaGs(mij
MJ0) T D Di [_
7. E [log p()) J(T 1) (1)($1) log( (2)) (51/$2 E ET- g(bje, azt).
8. E [log(())] = (K - (1b(m)) - log(r2)) + (11/12 - 1) K-1 g(bx, āz).
9. E [log((a)] (am( bm)) + (a.. ()((((81) - log(s2)) b 81 /82, where C. C.).) is the normalizing constant of Gamma distribution.
10. E [log p()] la, b, + (a. ()(((r)))))))))))))))))))))))))) - log(r2) byr1/r2
The second term decomposed into the following six components:
1. E [log ((t) = E =1 E Eeit log(Sjst)
2. E [log c(k)] Σ/ Dk Piuk log(pjex)-
k
3. E [log q()) = Σ ET (log(Cu (āzz, b,t)) ) + (āfe - 1)g((ze, + (b,e - 1)9(bse. a,d), where Ca(+) (-) is the normalizing constant of a Beta distribution.
22

---

Global-Local Dirichlet Process
4. E llog (()) Ek=1 (log(Co (aps bx)) (āx 1)g((z) bk)) (bx 1)g(bks ax))
5. E log QQI'' (pN Antt] = Ek=n [0.5ch + +0.5p(log(aa/2t)-1 ( (g(A))) where H (g(As)) is the entropy
of a Wishart distribution.
6. E log -
A113)] = =E Σt= (0.5 (1) + 0.5pp (log() - H (g(A
(81 1)((((81) llo((s)) 81, where Cal.) is the normalizing constant of 7. E [oo({(a))] log(Ca (51, 82))
Gamma distribution.
8. E [log((())) = (rg(C,(r,,22) (r1 ()((())) log(r2)) rl.
G. Simulations
G.1. Comparison of MCMC and VI for the GLocal DP
First, we conduct simulations comparing the clustering accuracy of the GLocal DP using the MCMC-based blocked Gibbs sampler and the VI-based algorithm. Throughout the simulations, we assume that there are three groups or populations. We consider a simulation setting in which the shared variables across the populations are five dimensional. Furthermore, we assume that there are two, three, and four local variables for populations 1, 2, and 3, respectively. We generated the data from,
Bgj ~ (Jiaat (4)) (x)),
(G.1)
where,
41
(1 (191)= = Σ
Jyt (a/h) peit Agt
(G.2)
f2(2/) (b)) = Σ (aNa (x,
(G.3)
Lt, 3, Le 5, Lt, = 4, and Ly = 6. The true parameters and the true mixture weights corresponding to the local variables are drawn from,
Here P1 = 2, = P3 = =
(pg, Arr ~
a ~ Gamma 225, 1), म, ~ a//
for j = 1,,,3, and = I,...: Lnp- The true local indicator lis is drawn from a multinomial distribution with class probabilities n, for j 1,2, 2, Similarly, the true parameters and mixture weights corresponding to the global variables drawn from,
(ms, AR) NW 5.1,5 7 Gammaa 1), B Dir(7/ Lgg Lg),
for k 1.....Aa Lgv where p = 5. The true latent indicator k3s is drawn from multinomial distribution with the class probabilities B, for f = 1.... Lg- We considered several sample sizes for the three populations. In particular, we considered for all j = 2,2 nj = 100, 200, and 500.
We ran our MCMC sampler for 50,000 iterations. The first half of the iterations were discarded as burn-in, and posterior
samples were retained at every 25th iteration after burn-in. As for the VI approach, we considered A(t 1, < 10-5 as a stopping rule to define the convergence of the ELBO. Although the discrepancy between the variational distribution and the target posterior is reduced at each iteration, there is no guarantee that the CAVI algorithm will converge to global optimum, rather it will likely obtain local solution depending on the initialization. Hence, we executed 20 distinct runs of the algorithm with different starting points, keeping the one with the highest ELBO to draw the inference.
Depending on the computational strategy, the posterior point estimates of the clusters were obtained using different procedures. For the MCMC output, we estimated the clusters by minimizing the variation of information loss (Wade
23

---

Global-Local Dirichlet Process
& Ghahramani, 2018). When dealing with the VI approach, the algorithm returns the optimized variational parameters corresponding to the cluster assignment probabilities, i.e., Pjak= = q'kzs k) and Ejet q" (lz+ -). Hence, in this case, the clusters were estimated as
kyz = arg max Pyck and Ey: = arg max Ejst k=l....KK
t=l...,
jo,..,,,.,JJJ 1,..., T, and i 1,.... Rz. We assessed the accuracy of the estimated clusters using the adjusted Rand index (ARI, Hubert & Arabie, 1985) between the posterior point estimate and the true cluster. Figure 6(a) shows the distribution of the ARIs, over 50 simulated datasets obtained by MCMC based Gibbs sampler and VI for several choices of truncation levels, while keeping the sample size fixed at n 200 for all = 2, 3. Similarly, Figure 6(b) shows the distribution of ARIs for varying sample sizes in each group (N denotes the total sample size), and truncation levels fixed at 30. Clearly, the clustering performance from the two methods are comparable.
Next, we consider the the need for a variational inference approach over the MCMC based method. We compare the computational cost of the two algorithms. In partieular, we compare the memory usage and the computing time by both algorithms. For the MCMC methods, the need to store the entire Markov chains (at most, excluding the burn-in and post thinning) can raise issues with memory allocation. On the contrary, VI methods only require storing the optimized parameters. Figure 7 shows the computational time and memory used for varying truncation levels and sample sizes, as before. For the MCMC sampler, computation time denotes the total run time for 50,000 iterations. Contrarily, for the VI based approach, we report the maximum individual run time obtained over the 20 runs for each dataset as the computational cost of the CAVI. Clearly, Figure shows significant gains in computational time and resource utilization by the VI approach in comparison to the MCMC based approach.
G.2. Comparison of GLocal DP with DP Mixture Model and GMM
In the main manuscript, we presented the comparison of clustering accuracy of the GLocal DP with GMM at the local-level, when the global variables were dimensinalss.......... In this subsection, we further compare the clustering accuracy with a DP mixture model (DPMM, Escobar & West, 1995) applied separately to each group. Specifically, for the simulation study presented in the main manuscript-where three global variables are shared across the three populations, and each population has two, three, and four local variables, respectively-wwe -e fit a separate DPMM for each group, incorporating both global and local variables. As in the main manuscript, we varied the degree of separation in the local variables while keeping the sample size fixed at ny = 200 for all j = 2,2.3. The DPMM inference was conducted using an MCMC sampler run for 50,000 iterations, with the first half discarded as burn-in. After burn-in, posterior samples were retained at every 25th iteration. For cluster estimation under the DPMM, we followed the approach of minimizing the variation of information loss (Wade& & Ghahramani, 2018). As in our original analysis, also evaluated GMM separately on each group and applied our proposed GLocal DP, which jointly models both global and local variables. The evaluation was conducted over 50 independent replications, and clustering accuracy was assessed using the ARI. Figure 8 demonstrates that across different levels of separation in the local variables, GLocal DP achieves clustering accuracy that is either comparable to or superior to that of the DPMM in terms of local-level clustering within each group.
G.3. Clustering Performance of GLocal DP Using Additional Metries
In the main manuscript, we evaluated the clustering accuracy of the GLocal DP by comparing it to the HDP at the global-level and separate GMMs applied to each group at the local-level, using the ARI as the evaluation metric. In this subsection, we complement that analysis by incorporating two additional clustering accuracy measures: Normalized Mutual Information (NMI: Strehl & Ghosh, 2002) and Purity (Manning et al., 2008). Specifically, we compare the global-Ievel clustering performance of GLocal DP to that of HDP applied solely to the global variables, and the local-level clustering performance of GLocal DP to that of separate DPMMs fitted independently within each group. Figure 9 illustrates the superior clustering performance of ooca DP compared to HDP at the global-level, as measured by both NMI and Purity, across varying degrees of separation in the local variables. Similarly, Figure 10 compares the clustering performance of GLocal DP with that of separate DPMMs applied to each group at the local-level. The results demonstrate that, across different levels of separation in the local variables, GLocal DP achieves clustering accuracy that is either comparable to or superior to that of the DPMM, as assessed by both NMI and Purity.
24

---

Global-Local Dirichlet Process
Global-level
Local-level
1.00
0.75
0.50
04
0.00
1.00
M Method
0.50
0.25
0.00
1.00
0.75
0.50
QE
0.00
Population Populátion Population
Populátion Population Population
Varying Truncation Levels. Sample size fixed 200 per population.
Global-level
Local-level
0.75
0.25
1.00
Method VI
ARI
0.50
600
0.00
0.75
OCSIL M
0.25
Populátion Populátion Populátion
Populátion Populàtion Population
(b) Varying Sample Size. Truncation is at I| 30.
Figure 6. Accuracy of global-level and local-level clustering. Each panel shows the distribution of the ARI obtained across the 50 replications, for each configuration. The colors correspond to the algorithm.
G.4. Comparison of GLocal DP with HDP and G.M
In the main manuscript, we presented the comparison clustering accuracy of the GLocal DP with the HDP at the global-level and GMM at the local-level, when the global variables were three-dimensional. In this subsection, we further compare the clustering accuracy when the global variables were five-dimensional. As before, assumed that there were two, three, and four local variables for populations 2, and 3, respectively. We generated the data from (G.1) and as before, we assumed,
(Has Aze (7 Tp.)
(G.4)
25

---

Global-Local Dirichlet Process
Compubetion times (seec)
3000
Computation limes (seec)
1500
1000
500
Method
Method
3 MCMC
MCMC VI
20
CSINO pween Avtuungy
20
10
10
20
30
300
1500
sample
(a) Varying Truncation Levels.
Varying Sample Size.
Figure 7. Top row: distributions of the computing time (in seconds) over the 50 replications for the two algorithms, for cach configuration. Bottom row: distributions the memory usage MB) over the 50 replications the two algorithms, each configuration. The colors correspond to the algorithm.
1.00
0.75
0.50
Low
0.25
0.00
1.00
0.75
Moderate
Method
ARI
GLocal DP
0.50
T-----0
DPM
0.25
GMM
0.00
1.00
0.75
High
0.50
0.25
0.00
Population 1
Population
Population 3
Figure 8. Comparison of clustering accuracy of GLocal DP with DPM and GMM at the local-level with varying separation in the local variables.
a Gamma(25, 1), n, Dir(a/Le,.....qu
for 1,2,3, and = .,..., Ley- Furthermore, >7 is precision parameter corresponding the local variables. Similarly, the true parameters and mixture weights corresponding the global variables drawn from,
AB) NW 15 1pp
(G.5)
7 Gammaa25, 1), B Dir(y/ Lgg... Lg),
26

---

Global-Local Dirichlet Process
1.00
1.0
0.75
Low
0.8
0.25
0.00
0.6
1.00
1.0
0.75
Moderata
Methed
Method
E GLocal DP
0.8
GLocal DP HDP
Fl HDP
0.25
0.00
0.6
1.00
1.0
0.75
0.8
0.25
0.00
0.6
Population
Population
Population 3
Population 1
Population
Population
(a) NML
(b) Purity.
Figure 9. Comparison of clustering accuracy of GLocal DP using the metrics (a) NMI and (b) Purity with HDP at the global-level.
1.00
1.0
0.8
0.50
0.25
0.4
0.00
1.00
1.0
Modwott
Method
0.8
Method
0.50
GLocai
GLocal DPM
0.25
0.4
0.00
1.00
T
1.0
0.8
UPT
UPM
0.50
0.25
0.4
Population 1
Population
Population 3
Population 1
Population
Population
aa) NMI.
(b) Purity.
Figure Comparison of clustering accuracy Gloccal using the metrics (a) NMI and (b) Purity with DPMM at the local-level.
for &: 1,..., I. where p and > is precision parameter corresponding the global variables. For simulations. set Àg 1 in (G.5), making the separations between the global variables to be low. Furthermore, varied the degree of separation in the local variables for the three populations by varying the local-level precision parameter A 0.1, 0.05, 0.01 in (G.4), We considered for all j = 1, 2, the sample size, n, = 200. As before, HDP was applied to the global variables only whereas GLocal DP was applied to both global and local variables. Furthermore, we looked at the accuracy of local-level estimated clusters of GLocal DP and the estimated clusters obtained from a separate GMM for each group and all simulations were replicated 50 times. For the HDP, we ran the sampler for 50,000 iterations. The first half of the iterations were discarded as burn-in, and posterior samples were retained at every 25th iteration after burn-in. For the VI approach of GLocal DP, considered A(t 1, 10-5 stopping rule define the convergence of the ELBO. As before, executed 20 distinct runs of the algorithm with different starting points, keeping the one with the highest ELBO draw the inference. We assessed the accuracy of the estimated clusters using the ARI between the posterior point estimate and the true cluster.
As before, Figure shows that the clustering performance of the proposed GLocal DP was better than HDP and the clustering accuracy improves with the increasing separation in the local variables. Furthermore, at the local-level the GLocal
27

---

in (G.5), Global-Local Dirichlet Process
1.00
1.00
0.75
0.75
0.25
0.25
0.00
0.00
1.00
1.00
0.75
Method
0.75
Method
0.50
GLocalDP
GLocal DP GMM
0.25
HDP
0.25
0.00
0.00
1.00
1.00
0.75
0.75
0.50
0.25
0.25
0.00
Populátion 1
Population
Population 3
Population 1
Population 2
Population
Global-level clustering.
(b) Local-Ievel clustering.
Figure 11. Five-dimensional Global Variables: Comaprison of Clustering Accuracy of GLocal DP with (a) HDP at the global-level and (b) GMM the local-level with varying separation the local variables.
DP clustering accuracy is higher than a GMM on each group separately.
Next, we varied the sample sizes in each group, i.e., for j = 1, 2, the sample size, nj = 100, 200, and 500. fixed the degree separation the global and local variables for the three populations. In particular, we set À allowing moderate separation in the global variables. Furthermore, we set Åg 0.1 in (G.4), allowing low separation the local variables. We naively refer to the separation of the local variable by the level of information contains e.g., "Low" level of information in local variable corresponds to low separation in the local variable. Even in this case, when the local variables contain "low information", Figure 12(b) shows that the clustering accuracy of the GLocal DP is either comparable to or better than the HDP at the global-level across sample sizes. This underscores the importance of incorporating local variables, when available to improve clustering accuracy, further highlighting that our model is different than the HDP at the global-level. Furthermore, our joint clustering global and local variables facilitates information sharing across variable types, enhancing clustering performance the local-level. Specifically, the GLocal DP demonstrates superior clustering accuracy compared applying GMM independently each group, with accuracy further improving the sample size increases.
1.00
1.00
0.75
IN
0.75
0.50
OBE S
0.25
0.25
0.00
0.00
1.00
1.00
0.75
Method
0.75
000 N
Method
0.50
009
GLocalDP HDP
GLocal DP GMM
0.25
0.25
0.00
0.00
1.00
1.00
0.75
0.50
0001
00ST
0.25
0.25
0.00
0.00
Population 1
Population
Population 3
Population 1
Population 2
Population
Global-level clustering.
(b) Local-level clustering.
Figure 12. Varying Sample Size: Comaprison of clustering accuracy of GLocal DP with (a) HDP at the global-level and (b) GMM at the local-level.
28

---

Global-Local Dirichlet Process
G.S. Impact of the local variables in clustering
In this subsection, we examine the impact of local variables on clustering performance. In the main manuscript, we emphasized that our method differs from the HDP, even at the global-level. To illustrate this, we compare clustering accuracy when the global variables are two-dimensional. We assumed that there were one, two, and three local variables for populations 1, 2, and 3, respectively and we generated the data from (G.1). Furthermore, the true parameters and the true mixture weights corresponding to the local variables are drawn from (G.4) with ' = 0.01, resulting in high separability among the local variables. Similarly, the true parameters and mixture weights corresponding to the global variables are drawn from (G.5) with Ag = 1, making the separations between the global variables to be low. All other simulation details were same as in Section G.4. Figure 13(a) shows that in the presence of highly overlapped global variables, GLocal DP is able to identify clusters with perfect accuracy. In contrast, HDP struggles to distinguish these overlapping clusters, as shown in Figure 13(b). The group-specific local variables are shown in Figure 14 along with the estimated local-level clusters, which further show the high separability of the local variables. This separation of local variables aids in identifying overlapping clusters of global variables, underscoring the significance of incorporating group-specific local variables, when available, to enhance the clustering of shared variables.
G.6. Comparison with Versatile Hierarchical Dirichlet Process
In the main manuscript, we emphasized that, despite sharing similar modeling objectives, the formulation of the vHDPMM proposed by Dinari & Freifeld (2020) is fundamentally different from our proposed GL DP mixture model. The VHDPMM adopts a hierarchical specification in which global variables are modeled first, followed by local variables that are conditionally dependent on the global cluster assignments. In contrast, the GLocal DP is defined jointly leading to a more flexible model specification. Under our formulation, for any two distinct observations i and i' within the same group j. if tyy = Ejv. then it follows directly that Aiyt, k.t,..; that is, if the observations are assigned to the same local cluster, they necessarily share the same global cluster. This dependency arises naturally from the joint modeling structure. In the vHDPMM, however, the local clusters are defined conditionally on global assignments, and this hierarchical construction imposes the restriction that for any group j. if i E = and i' E x where k f k', then the two observations and i' cannot have the same global cluster, even if they share the same local feature. Furthermore, our model exactly reduces to the HDP in the absence of local variables for all the groups. However, the vHDPMM, even in the absence of local variables for all the groups, is not exactly the HDP mixture model. Finally, the two models differ in their approaches to posterior inference. We develop scalable variational inference algorithm, while the vHDPMM employs split- merit based MCMC sampling scheme.
Using the cosegmentation example presented in Dinari & Freifeld, 2020, for an image j. if two pixel observations i and i' share the same local cluster corresponding to spatial location, GLocal DP highlights that they automatically share the same global cluster (given by the RGB colors). However, vHDPMM first clusters the observations i and i' into global cluster (given by the RGB colors) and conditional on same global cluster, models the local clusters (given by spatial locations). This, we feel is restrictive (also possibly counter-intuitive in the cosegmentation example) and our model provides a natural method of estimating clusters, both corresponding to the global- and local-level. To highlight this, we performed some simple simulation studies.
First, we generated data from the vHDPMM model with three distinct groups or populations, setting the sample size in each group to n, = 100, for j = 1, 2. The global variables were drawn from a six-component trivariate Gaussian mixture model. While our VI-based approach is designed to accommodate varying dimensionality of local variables across different populations, the publicly available implementation of the vHDPMM model is restricted to settings where the local variables share common dimensionality across all populations. To ensure comparability under this constraint, we simulated two local variables for each group from a five-component mixture of bivariate Gaussian distributions. Additionally, we considered setting where both the global and local clusters were well separated. We then applied both the vHDPMM and the proposed GLocal DP model to the simulated data and evaluated clustering accuracy at both the global and local levels. As shown in Table 1, the vHDPMM performs well, as expected, given that the data were generated from its own model. Furthermore, the GLocal DP also achieves comparable clustering accuracy.
Next, we generated the data from the GLocal DP model with three groups/populatioon. All other simulation settings were the same as before. We then applied both the proposed GLocal DP and the vHDPMM to the same dataset evaluate and compare their clustering performance at both the global and local levels. The clustering accuracy results, summarized in Table 2, indicate that the GLocal DP model achieves high accuracy, as expected given that the data were generated from
29

---

Global-Local Dirichlet Process
Population
Population
Population
3
Clusters
9g
8
30
12
-1
-2
0
-2
X1
(a) GLocal DP.
Population
Population
Population
=0578
=
ARI = 0.382
cluster
-2
0
22
-1
-2
-1
0
X1
(b) HDP.
Figure 13. Two-dimensional Global Variables: Comparison of clustering Accuracy of (a) GLocal DP and (b) HDP when the global highly overlapped and local variables are separated. The colors indicate the estimated clusters. Adjusted Rand index is reported at top of each panel.
its own generative process. Notably, the vHDPMM also performs comparably well, yielding similar levels of clustering accuracy across both global and local levels. These results suggest that, in settings characterized by well-separated clusters both models exhibit robust and comparable performance, even when the data-generating mechanism aligns more closely with one model over the other.
Finally, we considered more challenging clustering scenario by increasing the sample size within each group and reducing the separation among clusters. Specifically, we set n, 200 for each group 1, 1, 3, and generated data from the VHDPMM with low separation in both global and local features. All other simulation settings remained consistent with previous experiments. We then applied both the vHDPMM and the proposed GLocal DP model to this dataset. As shown in Table 3, the Gl ooca DP model outperforms the vHDPMM in terms of clustering accuracy at both the global and local levels. Importantly, this superior performance is observed even though the data were generated from the vHDPMM. This advantage can be attributed to the joint modeling approach of the GLocal DP, which enables more coherent inference, in contrast to the hierarchical specification of the vHDPMM. These findings highlight the increased flexibility of the GLocal DP framework, demonstrating ability to accurately recover complex clustering structures even under misspecification of
30

---

Global-Local Dirichlet Process
Population 1 ARI =1
Population
ARI =1
2
Clusters
Clusters
5.0
0
1
x
-2
2
2.5
11
3
12
-4
0.0
19
0
50
100
150
200
-6
-3
Index
X1
Population
Population
ARI
ARI=1
3
2
Custters
Clusters
1
1
2
2
0
4
-1
4
1
17
17
-2
-3
-1
0
1
2
3
4
5
1
2
3
X1
X2
Figure Local-level clustering of local variables by GLocal DP. The colors indicate the estimated clusters. Adjusted Rand index reported at the top of each panel.
Table Comparison clustering performance vHDPMM and GLocal when the data are generated from vHDPMM. Sample size in each group n, 100 for = 1,2,3. 3. Both the global and local clusters have high separation. The accuracy of clustering was assessed using the adjusted Rand index (ART) hetween estimated and true cluster.
Global-Ievel Clusters
Local-level Clusters
Model
Group
Group 2
Group
Group
Group 2
Group
vHDPMM
1.00
0.98
1.00
0.59
0.79
0.44
GLocal DP
1.00
1.00
1.00
0.59
0.79
0.34
the true data-generating process.
H. Real Data Analysis with Versatile Hierarchical Dirichlet Process
In this section, we analyze the data from our motivational problem in the main manuscript with the vHDPMM. Recall that we analyzed four gastrointestinal (GI) tract cancers. namely esophageal, stomach, colon, and rectal cancers. The dataset included log-transformed gene expression for 60,483 genes across 173, 407,512,,, 512, and 177 patients with esophageal, stomach, colon, and rectal cancers, respectively. Since the publicly available implementation of the vHDPMM model is restricted to settings where the local variables share common dimensionality across all populations, we modified the selection the clinical variables. Particularly, we considered the number cigarettes smoked per day local variable for esophageal cancer, the number of positive lymph nodes for stomach cancer, the pre-operative and pre- treatment CEA for rectal cancer, and BMI as the local variable for colon cancer. After excluding patients with missing clinical data, the final sample sizes were 92, 363, 263, and 120 for esophageal, stomach, colon, and rectal cancers, respectively.
As before, we performed PCA on the combined gene expression data from the four cancers and retained the top ten PCs as the global variables. We ran the vHDPMM on the final dataset for 10, 000 iterations. Furthermore, as before, for visualization, we reduced the original combined gene expression data to two dimensions using the UMAP. Figure 15 shows
31

---

Global-Local Dirichlet Process
Table Comparison of clustering performance of vHDPMM and Gloocal DP when the data generated from the GLocal DP mixture model. Sample size in each group ni 100 for 1, Both the global and local clusters have high separation. The accuracy of clustering was assessed using the adjusted Rand index (ARI) between the estimated and true cluster.
Global-level Clusters
Local-Ievel Clusters
Model
Group
Group 2
Group
Group
Group 2
Group
vHDPMM
1.00
1.00
1.00
1.00
1.00
0.79
Gloa DP
1.00
1.00
1.00
1.00
1.00
1.00
Table 3. Comparison of clustering performance of vHDPMM and GLocal DP when the data generated from the vHDPMM.. Sample size in each group n, = 200 for = 1, Both the global and local clusters have low separation. The accuracy clustcring was assessed using the adjusted Rand index (ART) between the estimated and true cluster.
Global-level Clusters
Local-level Clusters
Model
Group
Group 2
Group 3
Group
Group 2
Group
vHDPMM
0.33
0.00
0.95
0.29
0.00
0.50
GLocal DP
0.36
0.82
1.00
0.24
0.50
0.48
the UMAP embeddings colored by the estimated global- and local-level clusters obtained from the vHDPMM, while Figure 16 shows kernel density plot of local variables, segregated by the estimated local-level clusters. Finally, as in the main manuscript, we plotted the Kaplan-Meier survival curves for each of the identified cancer subpopulations in Figure 17, colored by the estimated local-level clusters obtained from the vHDPMM.
32

---

Global-Local Dirichlet Process
Colon Cancer
Esophageal Cancer
Rectal Cancer
Stomach Cancer
lusters
-10
10
-5
UMAP;
(a) Global-level clusters.
Colon Cancer
Esophageal Cancer
Rectal Cancer
Stomach Cancer
10
lusters
I
-10
-10
-5
10
-10
-5
10
110
-5
0
10-10
-5
0
5
10
(b) Local-level clusters.
Figure 15. Global variables. (a) The colors indicate global-level clusters estimated by the vHDPMM. The colors indicate the estimated local-level clusters by the vHDPMM.
33

---

Global-Local Dirichlet Process
Colon Cancer
Esophageal Cancer
1.2
Cluster
2c
0.6
0.9
2e
Density
0.4
Cluster
0.6
3b
la
3d
1b
3g
0.2
0.3
3h
3k
0.0
0.0
100
200
BMI
Number of cigarettes per day
Rectal Cancer
Stomach Cancer
0.100
1.5-
0.075
Cluster
Cluster
Density
Density
1.0
0.050
3b
1d
3d
le
0.025
0.5
1f
0.000
0.0
50
100
150
200
20
40
CEA level
Number positive lymph nodes
Figure 16. Kernel density plot of local variables, colored by the local-level clusters estimated by the vHDPMM.
34

---

Global-Local Dirichlet Process
2c
+
35
31
Uster
20
3k
1.00
probabiitt
0.75
0.50
0.25
0.00
1000
2000
3000
Time
(a) Colon Cancer.
Overall Survival:
Rectal Cancer
Uster 24 2c 3b 3d
1.00
probability
0.75
0
1000
2000 Time
3000
4000
(b) Rectal Cancer.
probabiitty
0.75
0.25
0.00
400
800 Time
1200
1600
(c) Esophageal Cancer.
Cluster - 1e
Id
1.00
Survival probbbiity
0.75
0.50
0.25
0.00
1000
2000 Time
3000
4000
(d) Stomach Cancer.
Figure 17. Survival curves according to local-level clusters estimated by the vHDDMM for different cancers.
35

---

Global-Local Dirichlet Process
Acronym
Full Form
DP
Dirichlet process
GEM
Griffiths, Engen and McCloskey distribution
HDP
Hicrarchical Dirichlct process
nested DP
Nested Dirichlet process
Gooaa DP
Global-Local Dirichlet process
GMM
Gaussian mixture model
TCGA
The Cancer Genome Atlas
MCMC
Markov chain Monte Carlo
VI
Variational inference Coordinate ascent variational inference
CAVI
divergence
Kullhack-I Leibler divergence
ELBO
Evidence lower bound
ART
Adjusted Rand index
GI
Gastrointestinal
CEA
Carcinoembryonic antigen
BMI
Body mass index
PCA
Principal component analysis
PC
Principal component
UMAP
Uniform manifold approximation and projection
Table 4. List of acronyms and their full forms.
Notation
Definition
Zyi
Observation from group j
Local variables in group
02
Local parameters (factors)
Global variables shared across groups
0
Global parameters (factors) Random measure corresponding to group j Group-specific base mcasure for group
U
Common base measure
Measure product
(a 0%
Conditional distribution of local variables, conditional on local factors Conditional distribution of global variables, conditional global factors Local atoms corresponding to G, in group
Pa(a)
Vi
on
Shared global atoms corresponding to V Local-level cluster label of observation in group j Global-level cluster
observation in group j
G
Truncated measure corresponding Gy. truncated levels T and K Prior distribution of parameters 0 under GL.ocal DP prior
pbo.os (0)
pT. (0)
Prior distribution of parameters 0 under truncated GLocal DP prior Marginal distribution of data under GLocal DP prior
00,00 (m)
m T.K (x)
Marginal distribution of data æ under truncated GLocal DP prior
(0(00))
Posterior distribution of parameters 0 under GLocal DP prior
(0)ma)
Posterior distribution parameters under truncated GLocal prior
a(tsi (Esi h=ll
Variational distribution of fji. which is multinomial distribution
a(ksei (Pru
Variational distribution of kye, which a multinomial distribution
q(Vzs CR. bR)
Variational distribution of ux, which is beta distribution
g(uzzjāze, byc)
Variational distribution of Uyy. which beta distribution
g(a;
Variational distribution of ox, which is a gamma distribution
9(33r1,72)
Variational distribution of 7, which
gamma distribution
q(pe, Ma db, ct. Dx)
Variational distribution of (fAR A& is normal-Wishan distribution
q(1311 A3c; m,t, A,t, Cst, Dj.)
Variational distribution of [pyt, A,r3 normal- Wishart distribution
kj. = arg max -1........... Pzck
Optimal clusters corresponding to ky. obtained from VI
f,, arg max,-1.... Exce
Optimal local-level clusters obtained from VI
Table 5. Notation and corresponding definitions.
36


Reviews:
[{"review_id": "uzQLOM0qMY", "reviewer": "Reviewer_aX7e", "summary": "This paper addresses the problem of clustering grouped data where observations may include both group-specific and shared variables across groups.  The authors propose a novel Bayesian non-parametric approach called the global-local (GLocal) Dirichlet process. Unlike HDP, where clusters are derived from a common base measure, GLocal DP distributes clusters in each G_j across a global shared subspace and a group-specific local subspace. Where each cluster in G_j is characterized by both the \"shared\" distribution, which is modeled across global model, and the local distribution which is only modeled in the local group.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for all the comments. \n\n- **Comparison, differences and gap between this work and Dinari and Freifeld, UAI 2020.**\n\n**Response**: Dinari and Freifeld, 2020 discusses a setting in which data arises from multiple pre-defined groups and consists of variables that are shared across the groups (denoted by $x_{ji}$) as well as group-specific variables (denoted by $y_{ji}$), which is the same as in our case. They propose the versatile hierarchical Dirichlet processes (vHDP) to model this grouped data. However, there are some key differences in the modeling perspective of this method as compared to our proposed GLocal DP. Importantly, the two models are not special cases of each other. \n\nParticularly, their model is defined hierarchically by first modeling the shared global variables and then conditionally modeling the local variables conditional on the global clusters. Contrarily, our proposed GLocal DP mixture model is defined jointly for the global and local variables. Using our formulation, for two observations $i$ and $i' \\neq i$ in the same group $j$, if $t_{ji}=t_{ji'}$, then automatically, they share the same global clusters i.e., $k_{jt_{ji}}=k_{jt_{ji'}}$. However, this is not the case for the vHDPMM, where the local clusters for individuals $i$ and $i'\\neq i$ are defined conditional on their global clusters. This, we feel is restrictive (also possibly counter-intuitive as in the cosegmentation example of Dinari and Freifeld, 2020) and our model provides a natural method of estimating clusters, both corresponding to the global- and local-level. \n\n### **Simulation Results**\n\n#### **Data from vHDPMM**\n\nTo illustrate, we simulated data from the vHDPMM with three groups, where global variables followed a six-component trivariate Gaussian mixture and local variables followed a five-component bivariate Gaussian mixture. We then assessed clustering accuracy (measured by ARI between true and estimated clusters) for both global and local clusters using vHDPMM and our proposed GLocal DP.\n\n|  | **Global-level clusters** | | | **Local-level clusters** | | |\n|--|--|--|--|--|--|--|\n| **Groups**  | 1 | 2 | 3 | 1 | 2 | 3 |\n| **vHDPMM** | 1.00 | 0.98 | 1.00 | 0.59 | 0.79 | 0.44 |\n| **GLocal DP** | 1.00 | 1.00 | 1.00 | 0.59 | 0.79 | 0.34 |\n\nTable shows that the clustering accuracy of vHDPMM was good as expected, since the data was drawn from the vHDPMM. Furthermore, GLocal DP clustering accuracy is comparable to that from vHDPMM.\n\n#### **Data from GLocal DP**\n\nNext, we simulated data from the GLocal DP model with three groups, keeping all other settings unchanged. We then evaluated the clustering accuracy of both global and local clusters using the vHDPMM and our GLocal DP.\n\n|  | **Global-level clusters** | | | **Local-level clusters** | | |\n|--|--|--|--|--|--|--|\n| **Groups**  | 1 | 2 | 3 | 1 | 2 | 3 |\n| **vHDPMM** | -0.04 | -0.02 | -0.01 | 0.00 | -0.01 | 0.00 |\n| **GLocal DP** | 1.00 | 0.98 | 1.00 | 1.00 | 0.80 | 0.99 |\n\nTable shows that the clustering accuracy of GLocal DP was good as expected. However, the vHDPMM performed very poorly, both in terms of global- and local-level clustering. In summary, the GLocal DP offers greater flexibility, effectively adapting to different data-generating mechanisms while accurately capturing both global and local cluster structures.\n\n- **Question on scalability**\n\n**Response**: We conducted simulations to assess the scalability of our VI-based algorithm for the GLocal DP. Specifically, we considered two scenarios:  \n\n1. Varying the number of groups ($J$) while keeping the sample size per group ($n_j$) fixed.  \n2. Varying $n_j$ while keeping $J$ fixed.  \n\nIn the first scenario, we set $J = 5, 10, 100\\$ with $n_j = 100$, and in the second, we set $n_j = 100, 200, 500$ with $J = 10$. All other settings matched those in the main manuscript. We see that computation time scales approximately linearly in both cases (see Figure 5 in the attached link to PDF at the end). These results highlight the efficiency of our VI-based approach even for large datasets, with large number of groups and/or sample sizes in each group.\n\n- **Question on the kind of distributions supported**\n\n**Response**: Our methodology is highly flexible and can accommodate a wide range of distributions\nfor both the likelihood and prior, including those supported by DP (Ferguson, 1973) and the HDP (Teh et al., 2006). Furthermore, our VI-based approach can be readily generalized to settings where the data distribution belongs to the exponential family, as\noutlined in Blei and Jordan, 2006. Additionally, our model can integrate both continuous and/or discrete variables in both the global and local components.\n\nWe would like to **refer** the reviewer to the detailed point-by point responses to all questions, simulations, figures, and our responses to additional comments/suggestions provided in the PDF, [here](https://drive.google.com/file/d/159nPJz-V6EN1euHOhPXRqgG6MZ-uBKl7/view?usp=sharing).", "claims_and_evidence": "The claims in the paper are supported by the experiments.", "methods_and_evaluation_criteria": "While not familiar with the real datasets myself, both them and and the synthetic datasets looks reasonable and a good way to evaluate the model.\nThe one thing I am missing  in the benchmarks is comparing with a DPMM individually each group, in the same way that was done with the GMM, that seems to me a like an important comparison that should be added, and an HDPMM where you do not differentiate between the global and local parts.\nIn addition, I would propose to use NMI and Purity metrics as well, which in my opinions has high value when comparing clustering with a varying number of clusters, and they complement the ARI.", "theoretical_claims": "I have checked both GLocal DP proposed constructions, and the truncation approximation bounds error evaluation, and they all seems sound.", "experimental_designs_or_analyses": "Experiments look well, see above (`Methods And Evaluation Criteria`) for a missing but required comparison.", "relation_to_broader_scientific_literature": "The main contribution of the paper is proposing and addressing the global-local structure for multiple groups in a HDP-like setting, and while many of the discussed and relevant paper do not address this exact setting, there is a previous paper which proposed that setting prior, and addressed it in a similar fashion, see below.", "essential_references_not_discussed": "I would like to point the authors to `Scalable and Flexible Clustering of Grouped Data via Parallel and\nDistributed Sampling in Versatile Hierarchical Dirichlet Processes, Dinari and Freifeld, UAI 2020`.\n\nDinari and Freifeld discuss this exact setting in their paper, and propose a slightly different model to address it. They propose the Versatile HDP model, having a similar global-local structure, the difference is mostly around the way the group-wise weights are drawn. The larger difference is in the inference method, where the current paper propose a VI method, Dinari and Freifeld proposed a split-merge based MCMC inference method, with a large focus on per-group-size scalability.", "other_strengths_and_weaknesses": "The paper is well written, and the different model construction are very helpful for the understanding of the flow. The supplementary add significant value with the Gibbs sampler.\n\nThe main weakness of the paper related to the prior work by Dinari and Freifeld, which handles the same setting, and propose a slightly different model, reducing the novelty of the paper's main contributions (setting and model).\nA bit lesser concern is the scalability, the population sizes evaluated in the paper are small, HDP generally is used to tackle datasets with thousands of documents, each with thousands of words, this is kinda lacking here.\n\nFinally, see my comments on the evaluation methods for additional needed evaluations. Given that this paper does not compare versus other works, some additional work is needed to justify it, both in terms of results and practicality (running time, setup overhead, etc...)", "other_comments_or_suggestions": "I would suggest adding a notation table in the supplementary.", "questions_for_authors": "The following questions are issues that my greatly affect my score -\n\n1) Comparison, differences and gap between this work and Dinari and Freifeld, UAI 2020. \n2) Scalability, how does this method addresses datasets with large number of documents, or on the other side, very large groups (e.g. images for example)\n3) What kind of distributions are supported? Is the the same and DP and HDP, which are very flexible, or are there limitations? Both in general, and limitations between the global and local parts. Can the VI support this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fZx6QeNYvr", "reviewer": "Reviewer_f78g", "summary": "This paper considers the problem of clustering grouped data for which the observations may include group-specific variables in addition to the variables that are shared across groups.\n\nTo allow for these group-specific variables to aid in the clustering, the paper proposes a novel Bayesian nonparametric approach, termed global-local (GLocal) Dirichlet process, that models the ``global-local\" structure of the observations across groups. \n\nThe paper characterizes the GLocal Dirichlet process using the stick-breaking representation and the representation as a limit of a finite mixture model. The paper theoretically quantifies the approximation errors of the truncated prior, the corresponding finite mixture model, and the associated posterior distribution. \n\nThe paper develops a fast variational Bayes algorithm for scalable posterior inference, which is illustrated with extensive simulations and a TCGA pan-gastrointestinal cancer dataset.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the comment.\n\n- **The derivations seem straightforward, and the paper's impacts do not seem significant for ICML. What were the main technical innovations that overcame previously unresolved technical challenges?**\n\n**Response**:  In this paper, our contributions are as follows.\n\n1. We introduce a general Bayesian nonparametric framework, GLocal DP, for clustering grouped data by incorporating group-specific local variables. Most of the existing Bayesian or non-Bayesian clustering methods do not accommodate this idiosyncratic data structure. \n\n2. We formally define the GLocal DP mixture model and establish its finite mixture model representation. We further demonstrate that, in the limit, the finite mixture model converges to the GLocal DP mixture model. In practical applications, this implies that a suitably truncated finite mixture model closely approximates the infinite GLocal DP mixture model. However, to provide a rigorous quantification of this approximation, we derive explicit truncation error bounds in Propositions 1 and 2 of the main manuscript. Importantly, while our derivations may appear straightforward, they constitute a nontrivial extension of the results in Ishwaran and James, 2001 due to the presence of both local and global components in our model. \n\n3. In the Appendix of our main manuscript, we present a blocked Gibbs sampler for posterior inference in the GLocal DP, complementing the variational inference (VI)-based algorithm described in the main text. Since both inference approaches rely on a finite truncation of the infinite GLocal DP, our theoretical results provide practical guidelines for selecting appropriate truncation levels. Notably, existing Bayesian nonparametric models such as the hierarchical Dirichlet process (HDP, Teh et al., 2006) employ variational inference methods that assume a truncated variational family  (Teh et al., 2007; Wang et al., 2011), where the truncation level is typically chosen to exceed the expected number of clusters. While this heuristic is widely used, no established theoretical framework exists for determining truncation levels with explicit bounds on the approximation error. Since the GLocal DP reduces to the HDP in the absence of local variables across all groups, our truncation error bounds extend naturally to the HDP, thereby offering valuable theoretical guidance for selecting truncation levels in VI-based inference algorithms for the HDP.  \n\n4. We develop a VI-based algorithm for posterior inference in the GLocal DP and evaluate its performance through simulation studies. Our results demonstrate that the VI-based algorithm achieves clustering accuracy comparable to that of the MCMC-based Gibbs sampler while significantly improving computational efficiency in terms of both memory usage and runtime. Moreover, our VI-based inference algorithm exhibits high scalability with respect to both the number of groups and the number of observations per group.  \n\n5. We conduct additional simulations (found in the link to the PDF at the end), illustrating that incorporating local variables not only enables our model to accommodate complex data structures but also enhances the clustering performance of shared variables compared to methods that rely solely on shared variables. \n\n6. Additionally, our proposed method extends beyond the application to cancer genomics to a general grouped clustering framework, wherein the available data consists of important group-specific variables apart from the shared variables.\n\nIn summary, our novel Bayesian nonparametric model addresses a critical gap in the clustering of grouped data by explicitly incorporating group-specific idiosyncratic variables—an aspect not accounted for in existing literature. Furthermore, we establish theoretical truncation error bounds for the truncated GLocal DP prior and mixture model, offering a principled approach to selecting truncation levels in both MCMC- and VI-based inference algorithms. These theoretical results also extend to inference algorithms for the HDP, further enhancing their practical applicability. Finally, we develop a highly scalable VI-based inference algorithm for GLocal DP, which can be readily adapted for HDP-based models as well.\n\nWe would like to **refer** the reviewer to the additional simulations supporting the novelty and usefulness of our method along with other contributions of our work, provided in the PDF, [here](https://drive.google.com/file/d/18DGSpV1sx2qKS-JjD7deY1O6OFHOsyWV/view?usp=sharing).", "claims_and_evidence": "The results are sound.", "methods_and_evaluation_criteria": "The algorithm and experiments are sound.", "theoretical_claims": "The algorithm is sound.", "experimental_designs_or_analyses": "The experiments are sound.", "relation_to_broader_scientific_literature": "To allow for these group-specific variables to aid in the clustering, the paper proposes a novel Bayesian nonparametric approach, termed global-local (GLocal) Dirichlet process, that models the ``global-local\" structure of the observations across groups.", "essential_references_not_discussed": "The paper seems to have discussed essential references.", "other_strengths_and_weaknesses": "Strength: the paper uses group-specific variables to aid in the clustering\nWeakness: the derivations seem straightforward, and the paper's impacts do not seem significant for ICML", "other_comments_or_suggestions": "n/a", "questions_for_authors": "The derivations seem straightforward, and the paper's impacts do not seem significant for ICML.\n\nWhat were the main technical innovations that overcame previously unresolved technical challenges?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4uVqVg3cWk", "reviewer": "Reviewer_xxvp", "summary": "The article presents a new method for performing Bayesian nonparametric clustering on data sets with both global and local variables, i.e. data sets for which some variables are only observed for a subset of the individuals. The paper presents a novel formulation of a clustering model that allows for global clusters on the variables observed for all individuals, and local clusters discriminating within those observed in specific subsets of individuals.\n\nA variational bound is introduced to perform inference through optimisation, and experiments are run on simulated data and a pan-cancer genomics data set. Comparisons are made with the same model specification with inference performed with MCMC, alongside the Hierarchical Dirichlet Process (HDP) for global clustering and finite Gaussian Mixture Models (GMM) for the local clustering.\n\n##### update after rebuttal\n\nI am satisfied with the accessible parts of the authors' response. However, since they linked to external media which is not permitted in the openreview system, I have kept my score the same.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for all the comments and questions, responses to some of which are in the textbox below. \n\n- **Question related to allowing our algorithm to make use of information on presence/absence of some variable***\n\n**Response**: The presence of PSA measurement indicates prostate cancer. But since whether the patient has prostate cancer or not is a given information for our method, which determines the groups, we do not think it provides extra information for global clustering other than the PSA values themselves. However, as a possible future research, our model could be extended to achieve simultaneous clustering of groups and within-group observations. In such a context, the presence or absence of variables for some subgroups and not others can possibly be informative in identifying similar cancer subtypes. For example, the presence of PSA measurement represented as an indicator variable itself would be informative in identifying prostate cancer as a possible secondary cancer associated with the primary cancer of the given patient(s).\n\n- **Question related to relevance of existing methods for missing data in our setup**\n\n**Response**: Thank you for your question. While our framework does not follow conventional missing data mechanisms, the absence of certain clinical variables can be viewed as Missing Not at Random (MNAR) when missingness is tied to cancer type. For example, the absence of PSA measurements likely indicates the patient does not have prostate cancer, where PSA is a key diagnostic marker. More broadly, our approach accommodates scenarios where different groups have distinct relevant variables, which may appear missing not due to randomness but because they hold no relevance for those groups.\n\n- **Question related to treating the local variables as just *missing***\n\n**Response**: As highlighted previously, our framework does not follow conventional missing data mechanisms. Treating local variables as \"missing\" and imputing them may not yield meaningful grouped clustering. For example, in our pan-cancer analysis, CEA is a key biomarker for colorectal cancer but irrelevant for esophageal and stomach cancers. Imputing CEA before clustering could obscure biologically meaningful subgroupings. Thus, explicitly modeling local variables is essential for interpretable and valid clustering results.\n\n- **Question on influence of the proportion of local vs global variables**\n\n**Response**: This is an excellent question. Please see our detailed explanation in the link to the PDF at the end of the response.\n\n- **Question on inference for discrete variables**\n\n**Response**: Our GLocal DP model and VI-based algorithm can accommodate both continuous and discrete variables in the global and local components. As long as the parameters of discrete variables are continuous, our VI algorithm can optimize the ELBO using coordinate ascent. However, if the parameters were discrete, alternative discrete optimization methods or continuous relaxations might be needed, which could be explored in future research.\n\n- **Question on putting a gamma prior**\n\n**Response**:  We acknowledge that inconsistency in the number of clusters is an important and active area of research. Our intention is not to understate this challenge; rather, we adopted a non-informative gamma prior as one possible modeling choice to help mitigate the severity of clustering inconsistency.\n\n- **Question related to use of only Gaussian likelihoods in our model**\n\n**Response**: Our VI-based algorithm is designed for Gaussian distributions but is inherently flexible, supporting arbitrary likelihood and prior choices, including DP (Ferguson, 1973) and HDP (Teh et al., 2006). The Gibbs sampler in the Appendix is similarly extendable. Moreover, our VI approach generalizes to exponential family distributions (Blei & Jordan, 2006). The rationale for assuming Gaussian distribution for PCA components in the real data example are shown in the PDF. \n\n- **Question related to variables in-between local and global**\n\n**Response**: This is an excellent question. Please see our detailed answer in the link to the PDF at the end of the response.\n\n- **Question on including a single real data set**\n\n**Response**: Our team primarily works with genomics data, which is why we used TCGA as an example. Similar multi-omics datasets, such as ICGC, share the same structure, but we did not use them due to significant preprocessing requirements. Beyond genomics, grouped data with both global and local variables arise in fields like business, social sciences, and other applied domains, for which our model would provide a convenient framework for analysis, as discussed in the detailed response PDF.\n\nWe would like to **refer** the reviewer to the detailed point-by point responses to the questions, with additional simulations results, figures, and comments found in the PDF [here](https://drive.google.com/file/d/1Q-8oAM7nyEtGOURmVQF8E-ZTzwQl8IOi/view?usp=sharing).", "claims_and_evidence": "The methods are well-motivated and explained well. The theoretical grounding is reasonable coherent and rigorous. The experimental evaluation is a little light, relying on one simulated data example and one real data set, although each of them is performed thoroughly. The restriction to Gaussian likelihoods is quite strict, but will presumably be diversified in future work.", "methods_and_evaluation_criteria": "The benchmark data sets are appropriate for the data set but are limited to one simulated study and one real data set. It is possible that finding real data sets with the appropriate structure for this model specification is somewhat challenging.", "theoretical_claims": "The theoretical claims and results appear sound but I have not checked them thoroughly.", "experimental_designs_or_analyses": "The experimental design seems reasonable. The use of a single clustering index (Adjusted rand index) is a potential drawback, as there are multiple ways of assessing cluster algorithms. In the simulated data set, we have a \"ground truth\" to compare against, but this evaluation method will not generalise to real data.\n\nThere is no real comparison to competing methods for the real data set, which would be of interest. There is also no use of any kind of model fit index (BIC? ICL) to evaluate the model fit.\n\nThe assessment of the computational properties of the inference is clear and fairly thorough, e.g. computational time, convergence criteria etc.", "relation_to_broader_scientific_literature": "The broader scientific literature is fairly well described, including the historical 1973 Ferguson DP article followed by generalisations to the Hierarchical and Nested DPs. The more recent results concerning inconsistency of the DP for the number of clusters is acknowledged and some effort is made to counteract the potential resulting issues.", "essential_references_not_discussed": "There are no major references missing from a machine learning perspective. It is likely that the cancer/genomics references could be more thorough, but the ICML audience will not necessarily mind so much.", "other_strengths_and_weaknesses": "In general, this is an interesting methodological development that is well-motivated by an interesting application challenge. There are some extensions that might be necessary before this becomes generally usable for all problems of this nature, but the article itself is a reasonable contribution.", "other_comments_or_suggestions": "The quality of the written English is generally good.", "questions_for_authors": "The presence of variables for some subgroups and not others is going to be informative to the global clustering. Does the method aim to make use of this? E.g. PSA is only measured when prostate cancer is also suspected: Would the presence of a PSA measurement represented as an indicator variable itself be informative to global clustering? Are you allowing the algorithm to make use of this information?\n\nWhat is the relevance of existing methods for missing data? We are running into a sort of Missing at Random/Missing Completely at Random/Missing Not at Random kind of situation where the presence or absence of data is potentially going to be informative in itself. Where does this framework fall within the MAR/MCAR/MNAR set of assumptions?\n\nWhat would happen if you treated the local variables as just “missing” in a Bayesian framework for all the other (non-local) individuals?\n\nWhat is the influence of the proportion of local vs global variables? To consider two extremes:\n1. Two data sets bound together concerning two different illnesses, with different data collected for each of them, and the only shared variables being (e.g.) age and sex. What kind of structure will the method uncover here?\n2. Two heavily overlapping data sets with many shared variables and only one or two “extra” local variables per population. What happens here? Would a whole separate clustering be “discovered” for the limited extra variables? Could it be done in a way that builds on the dominant narrative from the shared variables?\n\nHow does the inference work for the discrete variables exactly? Can you use the gradients for the continuous variables and some other solution for the discrete ones? Is everything represented as a smooth continuous target function somehow? How robustly can you assess the convergence of a discrete optimisation algorithm?\n\nDoes putting a gamma prior on the concentration really solve the concentration/specification problem for DPs? Is the issue of inconsistency of the number of clusters for DPs generally overstated if a relatively simple solution exists?\n\nThe use of only Gaussian likelihoods is quite a hard constraint and likely to be false in reality much of the time. What is the relationship between the mixture component likelihood specification and the underlying clustering structure? Is it more important to specify the right data likelihood or the right underlying clustering structure? What kind of problems will you run into if you have the correct underlying clustering design but a misspecified likelihood? How easy would it be to adapt this method to non-Gaussian likelihoods? Why have you not done this? Can the PCA components in the real data example be safely assumed to be Gaussian?\n\nWhat if there are variables in-between local and global? i.e. there are a global variables shared between all individuals, local variables unique to specific subgroups, but also mid-level variables shared between some subgroups but not all of them? Would you have to introduce an extra component in equation 1? And extra indicator variables in addition to t and k? Could this become a more generic framework that could incorporate varying degrees of data granularity in that respect?\n\nWhy did you only include a single real data set? Is it a challenge to find data sets with this structure in practice? Is it hard to perform the relevant data formatting and similar to prepare potential target data sets for this analysis pipeline?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper proposes a variant of the classical HDP mixture model to handle a particular setting of grouped data. Rather than the standard setting of HDP, here the setting is such that there are both global variables as well as local (i.e., group-specific) idiosyncratic variables.

The paper is overall fairly well written, discusses an interesting problem, and proposes a principled solution. Experiments are, overall, convincing (but see remark below about the experiments).

A main issue that came up in the review is the existence of a previous paper, Dinari and Freifeld, UAI 2020. That paper, which proposed a very similar variant of HDP, was neither cited nor compared to in this submission.

**The AC has taken a very close look at both papers.**


The **setting** described in the present paper is **identical** to the one in Dinari and Freifeld.  
In terms of **modeling**, the Bayesian nonparametric hierarchical models in the current paper and in Dinari and Freifeld are not exactly the same. Seemingly, neither one generalizes the other, and there is a fairly subtle difference in their modeling approaches for capturing relations between global and local clusters (conditional in Dinari and Freifeld vs joint here).  
The **inference** methods, however, are different. While Dinari and Freifeld employed a split/merge-based MCMC method, here the approach is variational.


The Dinari and Freifeld paper decreases (but does not completely eliminate) the novelty of the present paper (e.g., the settings are identical and the idea to alter/extend the HDP to account for group-specific variables was already done in Dinari and Freifeld). Given that the settings are identical, and models are similar, the authors should have cited the paper, discussed it, and compared against it. However, it seems the authors were not originally aware of that paper, and in their rebuttal put a sincere effort into explaining the relation between the two papers, even though the empirical comparison they added (simulating data from each model and then comparing both models on it) is very limited. That said, probably not much more could have been expected during the short rebuttal period. In any case, the AC does not think that the existence of the Dinari and Freifeld paper implies that this submission should be rejected. 

Two reviewers -- including the one who brought up the Dinari and Freifeld paper -- support accpeting this paper. A third reviewer is more negative, but his/her review was less thorough than the two others, and that reviewer didn't engage in the author-reviewer discussion.  

**Taking all of the above that into account, the AC recommends accepting the paper. That said, in their camera-ready version, the authors are expected to discuss the Dinari and Freifeld paper, be transparent about the fact their setting was already introduced by Dinari and Freifed, clearly explain the nuanced differences between the models** (and, to a lesser extent, the difference between the inference methods, which are clearly different), and **provide an empirical comparison**. It is also **highly recommended** that this comparison be more thorough, including on real datasets, and not just on the simulated data as the authors did in their rebuttal.
Decision: Accept (poster)