# Impact of missing data on mixtures and clustering with illustrations in Biology and Medicine

C. Biernacki

with C. Boyer, G. Celeux, J. Josse, F. Laporte, M. Marbac, A. Sportisse, V. Vandewalle

The 24th Conference of the Romanian Society of Probability and Statistics
Bucharest, April 21-22, 2023
## Take home message

Missing data may change preconceptions

* **Mixtures:** EM has unexpected behaviour concerning degeneracy dynamic

* **Clustering:** the missing data pattern may convey some information on partition

These topics are in the research agenda of Statisticians since:

The larger the datasets, the more missing data may appear...
# Outline

## 1 Introduction

## 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data

## 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration

## 4 Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration

## 5 Concluding remarks
<figure><img src="image_1.png" /></figure>

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

Missing data: notations

■ $y = \{y_1 | \dots | y_n\}^T$: full dataset with $n$ individuals

■ $y_i = (y_{i1}, \dots, y_{id}) \in \mathcal{Y} = \mathbb{R}^d$ with individual $i \in \{1, \dots, n\}$

■ $c = \{c_1 | \dots | c_n\}^T \in \{0, 1\}^{n \times d}$: pattern of missing data for the full dataset

■ $c_i = (c_{i1}, \dots, c_{id}) \in \{0, 1\}^d$: pattern of missing data for individual $i \in \{1, \dots, n\}$

$$c_{ij} = 1 \Leftrightarrow y_{ij} \text{ is missing}$$

■ $y_i^{\text{obs}}$: the observed variables values for indiv. $i$ (and $y^{\text{obs}} = \{y_1^{\text{obs}} | \dots | y_n^{\text{obs}}\}^T$)

■ $y_i^{\text{mis}}$: the missing variables values for individual $i$ (and $y^{\text{mis}} = \{y_1^{\text{mis}} | \dots | y_n^{\text{mis}}\}^T$)
## Typology of the missingness mechanisms

* Missing completely at random (MCAR):

$$p(c|y; \psi) = p(c; \psi) \quad \forall y$$

* Missing at random (MAR):

$$p(c|y; \psi) = p(c|y^{\text{obs}}, \psi) \quad \forall y^{\text{mis}}$$

* Missing not at random (MNAR): the mechanism is not MCAR nor MAR
<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

Mixture model and clustering (Gaussian case)

■ Partition (K clusters): $z = (z_1|...|z_n)^T \in \{0,1\}^{n \times K}$ where

■ $z_i = (z_{i1},...,z_{iK}) \in \{0,1\}^K$

■ $z_{ik} = 1$ if $y_i$ belongs to cluster $k$, $z_{ik} = 0$ otherwise

■ Mixture model: $y_1,...,y_n$ are i.i.d. from the d-variate Gaussian mixture

$$f(y_i; \theta) = \sum_{k=1}^{K} \pi_k f_k(y_i; \alpha_k)$$

■ $\pi_k = p(z_{ik} = 1)$

■ $f_k(.; \alpha_k) = \phi(.; \mu_k, \Sigma_k)$ is the d-variate Gaussian distribution with mean vector $\mu_k$ and covariance matrix $\Sigma_k$

■ $\theta = (\pi_1,..., \pi_K, \alpha_1,..., \alpha_K)$ is the whole mixture parameter

■ Clustering: MAP principle from the mixture output to estimate the partition

And also: algorithm estimation (EM, SEM...) and model selection (BIC, ICL...)
## Outline

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data
<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

Degeneracy genesis: unbounded likelihood

■ d-variate Gaussian mixture

$$
f(y_i; \theta) = \sum_{k=1}^{K} \pi_k \frac{1}{\left(\frac{(2\pi)^{d/2}}{|\Sigma_k|^{1/2}} \exp\left(-\frac{1}{2}(y_i - \mu_k)^T \Sigma_k^{-1} (y_i - \mu_k)\right)\right)^{\frac{1}{2}}} \phi(y_i; \mu_k, \Sigma_k)
$$

■ Sampling: $y_1, \dots, y_n \stackrel{i.i.d.}{\sim} p(\cdot; \theta)$ without any missing data ($y^{\text{obs}} = y$)

■ Likelihood: $\ell(\theta; y) = \ln L(\theta; y) = \sum_{i=1}^{n} \ln f(y_i; \theta)$

$$
\text{particular center } \mu_2 = y_i \quad \Rightarrow \quad \lim_{|\Sigma_2| \to 0} \ell(\theta; y) = +\infty
$$

[Kiefer and Wolfowitz, 1956] [Day, 1969]
EM behaviour: illustration

<figure><img src="image_13.png" /></figure>

■ degeneracy may occur even when starting from large variances

■ convergence can be slow when far from the degenerate limit

■ convergence extremely fast near degeneracy
<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

EM behaviour: results

<figure><img src="image_20.png" /></figure>

$$u_0 = \left[ \frac{1}{p_{i0} k_0}, \{p_{ik_0}\}_{i \neq i_0} \right]$$

degeneracy of component $k_0$ at $y_{i_0}$

⇔

$$\|u_0\| \to 0$$

[Biernacki and Chrétien, 2003]

[Ingrassia and Rocci, 2009]

Proposition 1: Existence of a bassin of attraction

$$\exists \epsilon > 0 \text{ s.t. if } \|u_0\| \le \epsilon \text{ then } \|u_0^+\| = o\|u_0\| \text{ with probability 1.}$$

Proposition 2: Speed towards degeneracy is exponential

$$\exists \epsilon > 0, \alpha > 0 \text{ and } \beta > 0 \text{ s.t. if } \|u_0\| \le \epsilon \text{ then, with probability 1, } \\ |\Sigma_{k_0}^+| \le \alpha/|\Sigma_{k_0}| \cdot \exp(-\beta/|\Sigma_{k_0}|).$$
## Outline

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data
Detail from the illustration

<table><caption>Table : Data belonging to the degenerated component.</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th></tr></thead><tbody><tr><th>1</th><td>211.00</td><td></td><td>0.09</td><td>30.75</td><td>151.98</td><td>4.94</td><td>14.27</td><td>27.24</td><td>217.13</td></tr><tr><th>2</th><td>196.86</td><td>0.02</td><td>0.09</td><td>28.59</td><td>82.06</td><td>2.87</td><td>7.97</td><td>27.66</td><td>200.75</td></tr><tr><th>3</th><td>144.00</td><td>0.12</td><td>0.05</td><td>19.65</td><td>70.43</td><td>3.58</td><td></td><td>7.57</td><td>160.37</td></tr><tr><th>4</th><td>172.52</td><td>0.13</td><td>0.04</td><td></td><td>192.22</td><td>5.12</td><td>19.32</td><td>32.19</td><td>174.93</td></tr><tr><th>5</th><td>121.00</td><td>0.17</td><td>0.09</td><td>24.44</td><td>144.47</td><td>5.91</td><td>22.02</td><td>10.59</td><td>141.77</td></tr><tr><th>6</th><td>223.00</td><td>0.12</td><td>0.08</td><td>33.10</td><td>197.01</td><td>5.95</td><td>30.45</td><td>12.96</td><td>252.48</td></tr><tr><th>7</th><td></td><td>0.17</td><td>0.23</td><td>34.22</td><td>94.35</td><td>2.76</td><td>31.28</td><td>13.88</td><td>180.61</td></tr><tr><th>8</th><td>303.00</td><td>0.06</td><td>0.04</td><td>22.57</td><td></td><td>4.54</td><td>21.83</td><td>5.72</td><td>321.65</td></tr><tr><th>9</th><td>250.00</td><td>0.09</td><td>0.09</td><td>29.64</td><td>180.76</td><td>6.10</td><td>26.14</td><td>13.96</td><td>280.12</td></tr><tr><th>10</th><td>391.00</td><td>0.06</td><td>0.01</td><td>35.78</td><td></td><td>7.41</td><td>22.13</td><td>28.11</td><td>400.99</td></tr><tr><th>11</th><td>176.00</td><td>0.09</td><td>0.08</td><td>20.59</td><td>79.71</td><td></td><td>18.23</td><td>9.58</td><td>191.99</td></tr><tr><th>12</th><td>145.00</td><td></td><td>0.11</td><td>21.22</td><td>82.46</td><td>3.89</td><td>20.30</td><td>6.17</td><td>162.51</td></tr><tr><th>13</th><td>124.13</td><td>0.13</td><td>0.11</td><td>20.59</td><td></td><td></td><td>18.46</td><td>9.12</td><td>134.89</td></tr><tr><th>14</th><td>103.00</td><td>0.16</td><td>0.29</td><td>23.75</td><td>78.26</td><td>3.29</td><td>22.32</td><td>8.12</td><td>124.98</td></tr></tbody></table>

■ Cvg. towards a degenerated component (no plateau of the log-likelihood)

■ Degeneracy relatively slow: log-likelihood linear according to the nb of it.

■ Number of points of the degenerated solution greater than the space dimension d (but the number of complete points lower than d)
<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

Intermediate conclusion on missing data

Like the complete data y case

■ Likelihood is unbounded

■ EM can be attracted by degenerate solutions

Unlike the complete data y case

■ Risk to consider a degenerated solution as valid

■ Risk of losing a lot of time in useless iterations

Statisticians should be aware of such dangerous EM behaviour...

... since missing data are more and more frequent
<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

Understanding degeneracy speed on a toy example

■ Univariate framework, no mixture, only one observed data: $y$

■ Maximum likelihood estimator (Unbounded likelihood!): $\mu = y, \sigma^2 = 0$

■ Suppose equivalently that $n-1$ data are unobserved (unchanged likelihood)

■ Here is one iteration of a (useless) EM algorithm (it. $q$)

$$ \mu^{(q+1)} = \frac{(n-1)\mu^{(q)} + y}{n} \quad \text{and} \quad \sigma^2(q+1) = \frac{(n-1)\sigma^2(q) + (y - \mu^{(q+1)})^2}{n} $$

Linear grow of the log-likelihood (have a look also when $n$ increases!)

$$ \ell(\theta^{(q)}; x) \sim -0.5q \log \frac{n-1}{n} $$

Geometrical convergence rate towards 0 for the variance

$$ \sigma^2(q) \sim \sigma^2(0) \left( \frac{n-1}{n} \right)^q $$
## Outline

### 1 Introduction

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data

### 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration

### 4 Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration

### 5 Concluding remarks
# Outline

## 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration
<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

Full mixed data: conditional independence everywhere<fnref n="1" />

The aim is to combine continuous, categorical, integer data, ordinal, ranking and functional data

$$y_1 = (y_1^{\text{cont}}, y_1^{\text{cat}}, y_1^{\text{int}}, \dots)$$

The proposed solution is to mixed all types by inter-type conditional independence

$$p(y_1; \alpha_k) = p(y_1^{\text{cont}}; \alpha_k^{\text{cont}}) \times p(y_1^{\text{cat}}; \alpha_k^{\text{cat}}) \times p(y_1^{\text{int}}; \alpha_k^{\text{int}}) \times \dots$$

In addition, for symmetry between types, intra-type conditional independence

Only need to define the univariate pdf for each variable type!

* Continuous: Gaussian

* Categorical: multinomial

* Integer: Poisson

* ...

<footnote n="1">¹MixtComp package: https://cran.r-project.org/web/packages/RMixtComp/index.html</footnote>
# Missing data: MAR assumption and estimation

## Assumption on the missingness mechanism

Missing At Randon (MAR): the probability that a variable is missing does not depend on its own value given the observed variables.

## Observed log-likelihood...

$$ \ell(\theta; y^{\text{obs}}) = \sum_{i=1}^{n} \log \left( \sum_{k=1}^{K} \pi_k p(y_i^{\text{obs}}, \alpha_k) \right) = \sum_{i=1}^{n} \ln \left[ \sum_{k=1}^{K} \pi_k \underbrace{\int_{y_i^{\text{mis}}} p(y_i^{\text{obs}}, y_i^{\text{mis}}; \alpha_k) dy_i^{\text{mis}}}_{\text{MAR assumption}} \right] $$
# SEM algorithm

(implemented in the MixtComp package, available on the CRAN repository)

A SEM algorithm to estimate $\theta$ by maximizing the observed-data log-likelihood

* Initialisation: $\theta^{(0)}$

* Iteration nb $q$:

* E-step: compute conditional probabilities $p(y^{\text{mis}}, z|y^{\text{obs}}; \theta^{(q)})$

* S-step: draw $(y^{\text{mis}(q)}, z^{(q)})$ from $p(y^{\text{mis}}, z|y^{\text{obs}}, \theta^{(q)})$

* M-step: maximize $\theta^{(q+1)} = \arg\max_{\theta} \ln p(y^{\text{obs}}, y^{\text{mis}(q)}, z^{(q)}; \theta)$

* Stopping rule: iteration number

## Properties: simpler than EM and interesting properties!

* Avoid possibly difficult E-step in an EM

* Classical M steps

* Avoids local maxima

* The mean of the sequence $(\theta^{(q)})$ approximates $\hat{\theta}$

* The variance of the sequence $(\theta^{(q)})$ gives confidence intervals
# Outline

3 Impact of MAR data on clustering

■ A model-based MAR approach

■ Biology study illustration
Prostate cancer data (with missing data)<fnref n="3" />

* **Individuals:** 506 patients with prostatic cancer grouped on clinical criteria into two Stages 3 and 4 of the disease

* **Variables:** *d* = 12 pre-trial variates were measured on each patient, composed by eight continuous variables (age, weight, systolic blood pressure, diastolic blood pressure, serum haemoglobin, size of primary tumour, index of tumour stage and histolic grade, serum prostatic acid phosphatase) and four categorical variables with various numbers of levels (performance rating, cardiovascular disease history, electrocardiogram code, bone metastases)

* **Some missing data:** 62 missing values (≈ 1%)

We forget the classes (Stages of the disease) for performing clustering

Questions

* How many clusters?

* Which partition?

<footnote n="3">³Byar DP, Green SB (1980): Bulletin Cancer, Paris 67:477-488</footnote>
<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

Two strategies in competition

■ Strategy "mice<fnref n="4" /> + MixtComp": MixtComp on the dataset completed by mice

> data.imp=mice(data)

> data.comp.mice=complete(data.imp)

■ Strategy "full MixtComp": MixtComp on the observed (no completed) dataset

Partition quality with K = 2

<table><thead><tr><th>Strategy</th><th>mice + MixtComp</th><th>full MixtComp</th></tr></thead><tbody><tr><td>% misclassified</td><td>12.8</td><td>8.1</td></tr></tbody></table>

<footnote n="4">⁴http://cran.r-project.org/web/packages/mice/mice.pdf</footnote>
Choosing *K* with the ICL criterion

<figure><img src="image_33.png" /></figure>

mice + MixtComp
*K* = 7

<figure><img src="image_34.png" /></figure>

full MixtComp
*K* = 2

... may lose some cluster information when imputation before clustering
<figure><img src="image_35.png" /></figure>

<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

Proposed MNAR models in clustering

Question we address now

Since MNAR is not ignorable, which distribution $p(c|y, z; \psi)$ to propose?

Hypothesis 1: conditional independence

$$p(c_i|y_i, z_{ik} = 1; \psi) = \prod_{j=1}^{d} p(c_{ij}|y_i, z_{ik} = 1; \psi)$$

Hypothesis 2: linear function within canonical link functions $\rho$

$$p(c_{ij} = 1 | y_i, z_{ik} = 1; \psi) = \rho(\alpha_{kj} + \beta_{k,j} y_{ij})$$

* $\psi = (\alpha, \beta)$ where $\alpha = (\alpha_{11}, \dots, \alpha_{1d}, \dots, \alpha_{K1}, \dots, \alpha_{Kd})^T \in \mathbb{R}^{Kd}$ and $\beta = (\beta_{11}, \dots, \beta_{1d}, \dots, \beta_{K1}, \dots, \beta_{Kd})^T \in \mathbb{R}^{Kd}$

* $\rho$ is the cdf of any continuous distribution (logit, probit)
A by-product zoology of MNAR models

<table><thead><tr><th rowspan="2"></th><th colspan="2">Effect on the<br>variable j</th><th colspan="2">Effect on the class<br>membership k</th><th rowspan="2">Nb parameters<br>Continuous</th></tr><tr><th>Depends<br>on j</th><th>Depends<br>on k</th><th>Depends<br>on j</th><th>Depends<br>on k</th></tr></thead><tbody><tr><td>MNARz<sup>j</sup>y<sup>k</sup></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>2Kd</td></tr><tr><td>MNARy<sup>j</sup></td><td>✓</td><td><b>X</b></td><td>✓</td><td>✓</td><td>(K + 1)d</td></tr><tr><td>MNARy<sup>k</sup>z</td><td>✓</td><td>✓</td><td><b>X</b></td><td>✓</td><td>K(d + 1)</td></tr><tr><td>MNARyz</td><td>✓</td><td><b>X</b></td><td><b>X</b></td><td>✓</td><td>(K + d)</td></tr><tr><td>MNARy</td><td>✓</td><td><b>X</b></td><td><b>X</b></td><td><b>X</b></td><td>d</td></tr><tr><td>MNARy<sup>k</sup></td><td>✓</td><td>✓</td><td><b>X</b></td><td><b>X</b></td><td>Kd</td></tr><tr><td>MNARz</td><td><b>X</b></td><td><b>X</b></td><td><b>X</b></td><td>✓</td><td>K</td></tr><tr><td>MNARz<sup>j</sup></td><td><b>X</b></td><td><b>X</b></td><td>✓</td><td>✓</td><td>Kd</td></tr></tbody></table>

Remarks:

■ MNARz<sup>j</sup>y<sup>k</sup> is the most complex model

■ MNARz, MNARz<sup>j</sup>: the only effect of missingness is on the class membership, $\psi = (\alpha_{11}, \dots, \alpha_{1d}, \dots, \alpha_{K1}, \dots, \alpha_{Kd})^T$, $p(c_{ij} = 1 | y_i, z_{ik} = 1; \psi) = \rho(\alpha_{kj})$

■ MCAR is a specific and simple case
<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

MNARz analysis: pattern c gives information on partition z!

Draw Bayes error of a MNARz model with two components and 20% of missing data

$\pi_k = 0.5, ||\mu_2 - \mu_1||$ varies, $\Sigma_1 = \Sigma_2 = I, |\alpha_2 - \alpha_1|$ varies

<figure><img src="image_41.png" /></figure>

Both $\mu_k$ and $\alpha_k$ act on the Bayes error
# Reinterpretation of the MNARz and MNARz<sup>j</sup> models as MAR

Commonly used in Machine Learning [Jones, 1996], [Little and Rubin, 2002], [Josse et al., 2019]

Mixture model for $y^{\text{obs}}$ and Bernoulli distribution for $C$

$$ \Leftrightarrow \text{MAR mixture model for } \tilde{y}^{\text{obs}} = (y^{\text{obs}}|c) $$

For example,

$$ y^{\text{obs}} = \begin{pmatrix} ? & 2.6 & 5 \\ \text{blue} & 1.9 & 4 \\ \text{red} & 2.3 & ? \end{pmatrix}, \quad c = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

then $\tilde{y}^{\text{obs}}$ is expressed as

$$ \tilde{y}^{\text{obs}} = \begin{pmatrix} ? & 2.6 & 5 & 1 & 0 & 0 \\ \text{blue} & 1.9 & 4 & 0 & 0 & 0 \\ \text{red} & 2.3 & ? & 0 & 0 & 1 \end{pmatrix}. $$

## Proposition 3: in terms of maximum likelihood

The maximum likelihood estimate associated to the dataset $\tilde{y}^{\text{obs}}$ under MAR model is the one associated to the dataset $y^{\text{obs}}$ under MNARz or MNARz<sup>j</sup> models.
<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

<figure><img src="image_44.png" /></figure>

# Identifiability

Previous works: [Teicher, 1963] (without NA), [Miao et al., 2016] (for MNAR data)

## Proposition 4

Assume that

1. The marginal mixture $\sum_{k=1}^{K} \pi_k f_k(y_j; \theta_k)$ is identifiable

2. There exists a total ordering $\leq$ of $\mathcal{F}_j \times \mathcal{R}$, for $j \in \{1, \dots, d\}$ fixed, where $\mathcal{F}_j = \{f_{1j}, \dots, f_{Kj}\}$ and $\mathcal{R} = \{\rho_1, \dots, \rho_K\} = \{\rho(\cdot; \psi_1), \dots, \rho(\cdot; \psi_K)\}$. The total ordering is s.t. $\forall k < l$, $F_k = \rho_k f_{kj} \preceq F_l = \rho_l f_{lj}$ implies

$$ \lim_{u \to +\infty} \frac{\rho_l(u)f_{lj}(u)}{\rho_k(u)f_{kj}(u)} = 0 $$

Then the mixture model with one of the MNAR* mechanisms is identifiable up to label swapping

All MNAR models are identifiable (or at least generically identifiable) for probit/logit
<figure><img src="image_45.png" /></figure>

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

Estimation procedure overview

■ Use EM or Stochastic EM (SEM) algorithms

■ MNARz and MNARz<sup>j</sup>: EM and SEM are very simple

■ MNARy*: the SE step requires a within Gibbs loop, sometimes involving itself a Sampling Importance Resampling (SIR)

<table><thead><tr><th></th><th colspan="2">EM</th><th colspan="2">SEM</th></tr><tr><th>MNARz<br>MNARz<sup>j</sup></th><th colspan="2">✓</th><th colspan="2">✓</th></tr><tr><th></th><th>Probit</th><th>Logit</th><th>Probit</th><th>Logit</th></tr></thead><tbody><tr><th>MNARy*</th><td>no closed form</td><td>no closed form, optim. pb</td><td>✓</td><td>require algorithms as SIR (costly)</td></tr></tbody></table>
Model selection

Can select between MCAR and MNAR* with any information criterion (BIC, ICL)

Even if the missing mechanism is ignorable for MCAR...

... need to model c to compare a MCAR and a MNAR model

CAUTION

■ It is just a selection between several proposed MNAR models

■ It is not deciding if missingness procedure is “genererically” MNAR or not
## Outline

### 4 Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration
<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

Hospital data description

■ Number of patients: $n = 5146$

■ Number of features: $d = 7$

■ Age

■ Size

■ Weight

■ Cardiac frequency

■ Hemoglobin concentration

■ Temperature

■ Minimum Diastolic and Systolic Blood Pressure

■ Percentage of missing data: 6.4%

Doctors are convinced that their missing data are MNAR
<figure><img src="image_51.png" /></figure>

<figure><img src="image_52.png" /></figure>

<figure><img src="image_53.png" /></figure>

ICL comparison

<figure><img src="image_54.png" /></figure>

■ MCAR, MNARy and MNARz are equivalent until $K = 3$

■ MNARz and MNARyz clearly indicate presence of an additional cluster ($K = 4$)

It seems to be an illustration of the effect of c through MNARz and MNARyz
## Outline

1. Introduction

2. Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data

3. Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration

4. Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration

5. Concluding remarks
<figure><img src="image_55.png" /></figure>

<figure><img src="image_56.png" /></figure>

<figure><img src="image_57.png" /></figure>

## To conclude

### Summary

* Statisticians should properly consider the potential missing data impact both from an algorithmic and from a modeling point of view

* EM: be careful about degeneracy which seems to exacerbated/masqued

* MAR: the minimal approach to be used by statisticians...

* MNAR: interest of the simple but meaningful model MNARz, link with usual methods

### Ongoing works

* EM: propose mechanism to identify/discard degenerate runs

* MNAR: extend to categorical, count and mixed data

Thanks!