# Levels Merging in the Latent Class Model

Christophe Biernacki

► To cite this version:

Christophe Biernacki. Levels Merging in the Latent Class Model. CFE-CMStatistics 2023, Dec 2023, Berlin, Germany. hal-04370783

HAL Id: hal-04370783

https://inria.hal.science/hal-04370783v1

Submitted on 3 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Levels Merging in the Latent Class Model

C. Biernacki

CFE-CMStatistics 2023, December 16-18 2023, Berlin (Germany)

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

Université
de Lille

<figure><img src="image_4.png" /></figure>
<figure><img src="image_5.png" /></figure>

Level merging could increase
clustering accuracy!

<figure><img src="image_6.png" /></figure>
# Outline

1. HD data clustering: explored and less explored cases

2. Our proposal: Latent Class Model by Levels Merging (LCM-LM)

3. Early numerical experiments

4. Concluding remarks

<figure><img src="image_7.png" /></figure>
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

Variety of high dimensional (HD) situations

Let a data set $\mathbf{x} = (\mathbf{x}_1, \dots, \mathbf{x}_n) \in \mathcal{X}^d$.

Well explored HD cases essentially rely on $d \to$ "HD-$d$"

■ Marketing: $d \sim 10^2$ (any kind of variables)

■ microarray gene expression: $d \sim 10^2-10^4$

■ Text mining $d \sim 10^4$ (count or categorical data)

■ SNP data: $d \sim 10^6$

■ Curves: depends on discretization but can be very high

■ ...

A less explored HD case: numerous levels for categorical data... $\to$ "HD-$m_j$"

■ Categorical data case: the $j$th variable has $m_j$ response levels

■ An underestimated HD case: some $m_j$s can be large

■ Ex: professional social categories ($m_j \sim 30$), US states ($m_j \sim 50$), French departments ($m_j \sim 100$), medical nomenclature ($m_j \sim 1000$)...

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>
<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

# HD issues in clustering

* Cluster analysis is one of the main data analysis method. It aims at estimating a K groups partition $z = (z_1, ..., z_n)$ of a data set $x = (x_1, ..., x_n) \in \mathcal{X}^d$

* Model-based clustering allows to reformulate cluster analysis as a well-posed estimation problem both for the partition $z$ and for the number $K$ of groups [McLachlan and Basford, 88]

* Model-based clustering has led practical successes, even in the challenging HD-d case [Biernacki and Maugis, 17]

* But less works on model-based clustering dedicated to the HD-$m_j$ case

## Have now a look at

* Classical model-based clustering for HD-$d$ methods

* The need of model-based clustering for HD-$m_j$ methods

<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>
# The general trade-off bias/variance as a guideline

## The fundamental statistical principle

Always minimize an error err between truth (z) and estimate (ŷ)

* Gap between true (z) and model-based ($Z_p$) partitions:

$$z^* = \arg \min_{\hat{z} \in Z_p} \text{err}(z, \hat{z})$$

* Estimation ŷ of z* in $Z_p$: any relevant method (bias, consistency, efficiency...)

* Fundamental decomposition of the observed error err(z, ŷ):

$$ 
\begin{aligned}
\text{err}(z, \hat{z}) &= \{\text{err}(z, z^*) - \text{err}(z, z)\} + \{\text{err}(z, \hat{z}) - \text{err}(z, z^*)\} \\
&= \{\text{bias}\} + \{\text{variance}\} \\
&= \{\text{error of approximation}\} + \{\text{error of estimation}\}
\end{aligned}
$$

<figure><img src="image_31.png" /></figure>
<figure><img src="image_32.png" /></figure>

<figure><img src="image_33.png" /></figure>

<figure><img src="image_34.png" /></figure>

<figure><img src="image_35.png" /></figure>

## Example for HD-d: reduce variance, accept bias

A two-component d-variate Gaussian mixture with intra-dependency:

$$ \pi_1 = \pi_2 = \frac{1}{2}, \quad \mathbf{X}_1|z_{11} = 1 \sim N_d(0, \Sigma), \quad \mathbf{X}_1|z_{12} = 1 \sim N_d(1, \Sigma) $$

* Each variable provides equal and own separation information

* Theoretical error decreases when d grows: $err_{theo} = \Phi(-||\mu_2 - \mu_1||_{\Sigma^{-1}/2})$

* Empirical error rate with the (true) intra-correlated model worse with d

* Empirical error rate with the (false) intra-independent model better with d!

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>
<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>

# Other classical ways for reducing variance in HD-d

## Dimension reduction

* In non-canonical space (PCA-like typically): Gaussian mixture of factor analysers [Ghahramani and Hinton, 97], [McLachlan et al., 03], [McNicholas and Murphy, 10], HD Gaussian models [Bouveyron et al., 07]

* In the canonical space (variable selection): Gaussian "variable selection" [Raftery and Dean, 06], [Maugis et al., 09a], [Maugis et al., 09b], [Sedki et al., 14]

* Model parsimony (constraints on model parameters): previous example, co-clustering [Govaert, 2011], [Biernacki et al., 23]

<figure><img src="image_42.png" /><figcaption>Continuous case</figcaption></figure>

<figure><img src="image_43.png" /><figcaption>Categorical case</figcaption></figure>

<figure><img src="image_44.png" /></figure>
<figure><img src="image_45.png" /></figure>

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

<figure><img src="image_48.png" /></figure>

HD-$m_j$: limit of the classical Latent Class Model (LCM)

■ When $\mathcal{X}$ corresponds to $d$ categorical variables (frequent situation), the $j$th one having $m_j$ response levels, the standard latent class model (LCM, [Goodman, 74]) is used

$$p(\mathbf{x}_j; \boldsymbol{\theta}) = \sum_{k=1}^{K} \pi_k \prod_{j=1}^{d} \prod_{h=1}^{m_j} (\alpha_k^{j,h})^{\mathbf{x}_j^{j,h}}, \quad \text{where } \boldsymbol{\theta} = (\pi, \boldsymbol{\alpha}), \pi = (\pi_k), \boldsymbol{\alpha} = (\alpha_k)$$

■ When some $m_j$s are large another kind of dimensionality curse appears

The standard latent class model is affected by such a curse of dimension paradigm since the number of free parameters if equal to $K - 1 + K \sum_{j=1}^{d} (m_j - 1)$

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>

<figure><img src="image_52.png" /></figure>

<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>

<figure><img src="image_56.png" /></figure>
<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>

<figure><img src="image_59.png" /></figure>

<figure><img src="image_60.png" /></figure>

# Outline

1. HD data clustering: explored and less explored cases

2. Our proposal: Latent Class Model by Levels Merging (LCM-LM)

3. Early numerical experiments

4. Concluding remarks

<figure><img src="image_61.png" /></figure>

<figure><img src="image_62.png" /></figure>

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

<figure><img src="image_65.png" /></figure>

<figure><img src="image_66.png" /></figure>

<figure><img src="image_67.png" /></figure>

<figure><img src="image_68.png" /></figure>
<figure><img src="image_69.png" /></figure>

<figure><img src="image_70.png" /></figure>

<figure><img src="image_71.png" /></figure>

<figure><img src="image_72.png" /></figure>

An early work for parsimony in HD-mj

[Lebret et al., 15] proposes four parsimonious versions of LCM. They correspond to an extension of the parameterization of Bernoulli distributions used by [Celeux and Govaert, 91] for clustering and also by [Aitchinson and Aitken, 76] for kernel discriminant analysis.

The basic idea is

$$ \alpha_k^j = (\alpha_k^{j1}, \dots, \alpha_k^{jm_j}) = (\beta_k^j, \dots, \beta_k^j, \gamma_k^j, \beta_k^j, \dots, \beta_k^j) \quad \text{with} \quad \gamma_k^j > \beta_k^j. $$

Since $\sum_{h=1}^{m_j} \alpha_k^{jh} = 1$, then $(m_j - 1)\beta_k^j + \gamma_k^j = 1$ and, thus, $\beta_k^j = (1 - \gamma_k^j)/(m_j - 1)$.

It leads to a very parsimonious (only $K - 1 + Kd$ parameters) but poor flexible model. We propose to extend this idea by a level partitioning (here an example with 3 level clusters)

$$ \alpha_k^j = (\alpha_k^{j1}, \alpha_k^{j2}, \alpha_k^{j3}, \alpha_k^{j4}, \alpha_k^{j5}) = (\beta_k^{j1}, \beta_k^{j2}, \beta_k^{j1}, \beta_k^{j3}, \beta_k^{j2}) $$

<figure><img src="image_73.png" /></figure>

<figure><img src="image_74.png" /></figure>

<figure><img src="image_75.png" /></figure>

<figure><img src="image_76.png" /></figure>

<figure><img src="image_77.png" /></figure>

<figure><img src="image_78.png" /></figure>

<figure><img src="image_79.png" /></figure>

<figure><img src="image_80.png" /></figure>
<figure><img src="image_81.png" /></figure>

<figure><img src="image_82.png" /></figure>

<figure><img src="image_83.png" /></figure>

<figure><img src="image_84.png" /></figure>

LCM-LM: a LCM extension for the case of numerous levels

■ **Idea:** constrain some levels to share the same parameter value (level partitioning)

■ **Notation:** for each variable $j$, $L_j$ clusters of levels $\mathbf{w}_j = (\mathbf{w}_j^1, \dots, \mathbf{w}_j^{m_j})$

■ **New model LCM-LM (Latent Class Model by Levels Merging):** from the full level clustering $\mathbf{w} = (\mathbf{w}_1, \dots, \mathbf{w}_d)$, the new LCM is defined by

$$ p(\mathbf{x}_i | \mathbf{w}; \vartheta) = \sum_{k=1}^{K} \pi_k \prod_{j=1}^{d} \prod_{\ell=1}^{L_j} \left( \frac{\beta_k^{j\ell}}{\sum_{h=1}^{m_j} w_h^{j\ell}} \right)^{\mathbf{x}_i^{jh}} $$

with $\beta = (\beta_1, \dots, \beta_K)$, $\beta_k = (\beta_k^{j\ell}; j = 1, \dots, d; \ell = 1, \dots, L_j)$ and $\vartheta = (\pi, \beta)$

■ **An interesting extreme case:** if $L_j = 1$ then LCM-LM acts as a variable selection (no effect of variable $j$ on the clustering)

LCM-LM is parsimonious with regards to the number of levels since its number of parameters is equal to $K - 1 + K \sum_{j=1}^{d} (L_j - 1)$

<figure><img src="image_85.png" /></figure>

<figure><img src="image_86.png" /></figure>

<figure><img src="image_87.png" /></figure>

<figure><img src="image_88.png" /></figure>

<figure><img src="image_89.png" /></figure>

<figure><img src="image_90.png" /></figure>

<figure><img src="image_91.png" /></figure>
<figure><img src="image_92.png" /></figure>

<figure><img src="image_93.png" /></figure>

<figure><img src="image_94.png" /></figure>

<figure><img src="image_95.png" /></figure>

## Parameter estimation

### Hint

Possible to reformulating the log-likelihood of LCM-LM as a classical LCM...

* LCM-LM $\Leftrightarrow$ LCM with the following new data $\mathbf{y}_\mathbf{w} = (\mathbf{y}_{\mathbf{w},1}, \dots, \mathbf{y}_{\mathbf{w},n})$, and where $\mathbf{y}_{\mathbf{w},i} = (\mathbf{y}_{\mathbf{w},i}^{j\ell}; j = 1, \dots, d; \ell = 1, \dots, L_j)$ with $\mathbf{y}_{\mathbf{w},i}^{j\ell} = \sum_{h=1}^{m_j} w_j^{h\ell} x_{ij}^{jh}$

* Parameter estimation: $L(\vartheta|\mathbf{w}; \mathbf{x}) = L(\vartheta; \mathbf{y}_\mathbf{w}) + c_\mathbf{w}$, where

$$c_{\mathbf{w}} = - \sum_{i=1}^{n} \sum_{j=1}^{d} \sum_{\ell=1}^{L_j} \mathbf{y}_{\mathbf{w},i}^{j\ell} \ln \left( \sum_{h=1}^{m_j} w_j^{h\ell} \right)$$

is a constant with regards to the parameter $\vartheta$

* Since $c_\mathbf{w}$ is independent of $\vartheta$ then use a classical LCM EM

<figure><img src="image_96.png" /></figure>

<figure><img src="image_97.png" /></figure>

<figure><img src="image_98.png" /></figure>

<figure><img src="image_99.png" /></figure>

<figure><img src="image_100.png" /></figure>

<figure><img src="image_101.png" /></figure>

<figure><img src="image_102.png" /></figure>

<figure><img src="image_103.png" /></figure>

<figure><img src="image_104.png" /></figure>

<figure><img src="image_105.png" /></figure>

<figure><img src="image_106.png" /></figure>
<figure><img src="image_107.png" /></figure>

<figure><img src="image_108.png" /></figure>

<figure><img src="image_109.png" /></figure>

<figure><img src="image_110.png" /></figure>

Related EM algorithm

As a consequence of the preceding remark, estimating $\vartheta$ can be performed by any classical EM algorithm [Dempster et al., 77] for LCM implemented in any classical existing package (for instance Rmixmod [Lebret et al., 15] or RMixtComp<fnref n="1" />).

Starting from $\vartheta^0 = (\pi^0, \beta^0)$, iteration $r > 0$ of this EM is expressed as:

E step: Conditional probability that individual $i$ arose from cluster $k$

$$t_{ik}(\vartheta^r) = \frac{\pi_k^r p(\mathbf{y}_{w,i}; \beta_k^r)}{p(\mathbf{y}_{w,i}; \vartheta^r)}.$$

M step: Updating the mixture parameter estimates by maximizing the expected complete log-likelihood

$$\pi_k^{r+1} = \frac{\sum_i t_{ik}(\vartheta^r)}{n} \quad \text{and} \quad (\beta_k^{j\ell})^{r+1} = \frac{\sum_{i=1}^n t_{ik}(\vartheta^r) y_{w,i}^{j\ell}}{\sum_{i=1}^n t_{ik}(\vartheta^r)}.$$

The stopping rule can rely on a plateau of the log-likelihood (equally $L(\vartheta; \mathbf{y}_w)$ or $L(\vartheta|\mathbf{w}; \mathbf{x})$) or a given iteration number. Several runs should be performed also from different starting parameters for avoiding local maxima traps.

<figure><img src="image_111.png" /></figure>

<figure><img src="image_112.png" /></figure>

<figure><img src="image_113.png" /></figure>

<figure><img src="image_114.png" /></figure>

<figure><img src="image_115.png" /></figure>

<figure><img src="image_116.png" /></figure>

<figure><img src="image_117.png" /></figure>

<figure><img src="image_118.png" /></figure>

<figure><img src="image_119.png" /></figure>

<figure><img src="image_120.png" /></figure>

<figure><img src="image_121.png" /></figure>

<figure><img src="image_122.png" /></figure>

<figure><img src="image_123.png" /></figure>

<figure><img src="image_124.png" /></figure>

<footnote n="1">¹https://cran.r-project.org/web/packages/RMixtComp/index.html</footnote>
<figure><img src="image_125.png" /></figure>

<figure><img src="image_126.png" /></figure>

<figure><img src="image_127.png" /></figure>

<figure><img src="image_128.png" /></figure>

## Model selection: criterion ICL

ICL criterion [Biernacki et al., 00], [Biernacki et al., 11]: just written as a penalization of the classical LCM ICL

$$ \mathrm{ICL}_{\mathrm{w}}^{\mathrm{LCM-LM}} = \ln p(\mathbf{x}, \hat{\mathbf{z}}_{\mathrm{w}} | \mathbf{w}) = \mathrm{ICL}_{\mathrm{w}}^{\mathrm{LCM}} + \underbrace{\mathrm{C}_w}_{\substack{\text{I} \\ \text{J}}} $$

where $\mathrm{ICL}_{\mathrm{w}}^{\mathrm{LCM}}$ is the ICL criterion for LCM. This latter is classically defined as follows

$$ \mathrm{ICL}_{\mathrm{w}}^{\mathrm{LCM}} = \sum_{k=1}^{K} \sum_{j=1}^{d} \left\{ \sum_{\ell=1}^{L_j} \ln \Gamma \left( \hat{n}_{\mathrm{w},k}^{j\ell} + \frac{1}{2} \right) - \ln \Gamma \left( \hat{n}_{\mathrm{w},k} + \frac{L_j}{2} \right) \right\} - \ln \Gamma \left( n + \frac{K}{2} \right) + \ln \Gamma \left( \frac{K}{2} \right) \\ + K \sum_{j=1}^{d} \left\{ \ln \Gamma \left( \frac{L_j}{2} \right) - L_j \ln \Gamma \left( \frac{1}{2} \right) \right\} + \sum_{k=1}^{K} \ln \Gamma \left( \hat{n}_{\mathrm{w},k} + \frac{1}{2} \right) - K \ln \Gamma \left( \frac{1}{2} \right), $$

where $\hat{n}_{\mathrm{w},k} = \#\{i : \hat{z}_{\mathrm{w},ik} = 1\}$ and $\hat{n}_{\mathrm{w},k}^{j\ell} = \#\{i : \hat{z}_{\mathrm{w},ik} = 1, y_{\mathrm{w},i}^{j\ell} = 1\}$.

As a consequence, the $\mathrm{ICL}_{\mathrm{w}}^{\mathrm{LCM-LM}}$ is itself easily calculated also.

<figure><img src="image_129.png" /></figure>
<figure><img src="image_130.png" /></figure>

<figure><img src="image_131.png" /></figure>

<figure><img src="image_132.png" /></figure>

<figure><img src="image_133.png" /></figure>

## Model selection: MICL criterion

MICL criterion (Maximum Integrated Complete-data Likelihood) [Marbac and Sedki, 15] is a variant of ICL avoiding multiple parameter estimation

$$ \text{MICL}_{\mathbf{w}}^{\text{LCM-LM}} = \ln p(\mathbf{x}, \mathbf{z}_{\mathbf{w}}^* | \mathbf{w}) \quad \text{with} \quad \mathbf{z}_{\mathbf{w}}^* = \arg \max_{\mathbf{z} \in \mathcal{Z}} \ln p(\mathbf{x}, \mathbf{z} | \mathbf{w}). $$

Then, the model $\mathbf{w}^*$ maximizing $\text{MICL}_{\mathbf{w}}^{\text{LCM-LM}}$ is retained: $\mathbf{w}^* = \arg \max_{\mathbf{w} \in \mathcal{W}} \text{MICL}_{\mathbf{w}}$. Finally a specific two step algorithm derived from [Marbac and Sedki, 15] can be used for performing this optimization (not implemented yet).

We do not detail this algorithm here but the idea is the following. Starting from a value $\mathbf{w}^0$ uniformly sampled in the corresponding space and then a value $\mathbf{z}^0$ being deduced from the MAP rule of the associated maximum likelihood estimate, iteration $s > 0$ of the algorithm is composed by the following two steps:

*   **Partition step** Fix $\mathbf{z}^{s+1}$ such that $\ln p(\mathbf{x}, \mathbf{z}^{s+1} | \mathbf{w}^s) \ge \ln p(\mathbf{x}, \mathbf{z}^s | \mathbf{w}^s)$.

*   **Model step** Fix $\mathbf{w}^{s+1}$ such that $\ln p(\mathbf{x}, \mathbf{z}^{s+1} | \mathbf{w}^{s+1}) \ge \ln p(\mathbf{x}, \mathbf{z}^{s+1} | \mathbf{w}^s)$.

<figure><img src="image_134.png" /></figure>
# Outline

1. HD data clustering: explored and less explored cases

2. Our proposal: Latent Class Model by Levels Merging (LCM-LM)

3. Early numerical experiments

4. Concluding remarks

<figure><img src="image_135.png" /></figure>

<figure><img src="image_136.png" /></figure>

<figure><img src="image_137.png" /></figure>

<figure><img src="image_138.png" /></figure>

<figure><img src="image_139.png" /></figure>

<figure><img src="image_140.png" /></figure>

<figure><img src="image_141.png" /></figure>

<figure><img src="image_142.png" /></figure>
<figure><img src="image_143.png" /></figure>

<figure><img src="image_144.png" /></figure>

<figure><img src="image_145.png" /></figure>

<figure><img src="image_146.png" /></figure>

# Prostate cancer data: description

* **Individuals**: 506 patients on the basis of petrial variates alone for the prostate cancer clinical trial data of [Byar and Green, 80], described by four categorical variables with various numbers of levels: performance rating (so-called PF, with 4 levels), cardiovascular disease history (HX, 2 levels), electrocardiogram code (EKG, 7 levels) and bone metastases (BM, 2 levels). There exists two stages of the disease (Stage 3 and Stage 4).

* **Some missing data**: 62 missing values (≈ 1%)

We forget the classes (Stages of the disease) for performing clustering

<figure><img src="image_147.png" /></figure>
<figure><img src="image_148.png" /></figure>

<figure><img src="image_149.png" /></figure>

<figure><img src="image_150.png" /></figure>

<figure><img src="image_151.png" /></figure>

# Prostate cancer data: LCM-LM vs LCM

* **EM-like package:** RMixtcomp<fnref n="2" /> for LCM categorical data and also dealing with missing values

* **Levels' distribution:** we see below the frequency of each level for all these variables (missing values are denoted by a “?” and appear below as a level, but mathematical speaking it is not a level obviously)

```
PF      HX      EKG      BM
?: 4    ?: 4    1 :168    ?: 4
1:450  1:289  5 :150    1:420
2: 37   2:213  6 : 75    2: 82
3: 13    3 : 51
4:  2    4 : 26
2 : 23
(Other): 13
```

* **LCM-LM proposal:** since level 4 of variable PF is under-represented (only 2 individuals), we propose to merge it with level 3 of the same variable

* **Results with K = 2:** great improvement both in ICL and error values

<table><thead><tr><th></th><th>LCM</th><th>LCM-LM</th></tr></thead><tbody><tr><td>ICL</td><td>-1728.4</td><td><strong>-1618.4</strong></td></tr><tr><td>error</td><td>49.2%</td><td><strong>28.8%</strong></td></tr></tbody></table>

<figure><img src="image_152.png" /></figure>

<footnote n="2">²https://cran.r-project.org/web/packages/RMixtComp/index.html</footnote>
<figure><img src="image_153.png" /></figure>

<figure><img src="image_154.png" /></figure>

<figure><img src="image_155.png" /></figure>

<figure><img src="image_156.png" /></figure>

# Outline

1. HD data clustering: explored and less explored cases

2. Our proposal: Latent Class Model by Levels Merging (LCM-LM)

3. Early numerical experiments

4. Concluding remarks

<figure><img src="image_157.png" /></figure>
<figure><img src="image_158.png" /></figure>

<figure><img src="image_159.png" /></figure>

<figure><img src="image_160.png" /></figure>

<figure><img src="image_161.png" /></figure>

Conclusion

■ Very promising preliminary numerical results

■ Need to still implement MICL

■ Extend simultaneously to high dimension both for the number of variables and the number of levels (merge co-clustering and LCM-LM?)

<figure><img src="image_162.png" /></figure>