# Impact of missing data on mixtures and clustering with illustrations in Biology and Medicine

Christophe Biernacki, Claire Boyer, Gilles Celeux, Julie Josse, Fabien Laporte, Matthieu Marbac, Aude Sportisse, Vincent Vandewalle

## ► To cite this version:

Christophe Biernacki, Claire Boyer, Gilles Celeux, Julie Josse, Fabien Laporte, et al. Impact of missing data on mixtures and clustering with illustrations in Biology and Medicine. ASA 2022 - Apprentissage Statistiques et Applications, Jun 2022, Poitiers, France. hal-03936781

HAL Id: hal-03936781

https://inria.hal.science/hal-03936781

Submitted on 12 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Impact of missing data on mixtures and clustering with illustrations in Biology and Medicine

C. Biernacki

with C. Boyer, G. Celeux, J. Josse, F. Laporte, M. Marbac, A. Sportisse, V. Vandewalle

June 23rd 2022, Poitiers
# Take home message

Missing data may change preconceptions

- **Mixtures:** EM has unexpected behaviour concerning degeneracy dynamic

- **Clustering:** the missing data pattern may convey some information on partition

These topics are in the research agenda of Statisticians since:

The larger the datasets, the more missing data may appear...
## Outline

### 1 Introduction

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data

### 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration

### 4 Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration

### 5 Concluding remarks
<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

Missing data: notations

■ $y = \{y_1|\dots|y_n\}^T$: full dataset with $n$ individuals

■ $y_i = (y_{i1}, \dots, y_{id}) \in \mathcal{Y} = \mathbb{R}^d$ with individual $i \in \{1, \dots, n\}$

■ $c = \{c_1|\dots|c_n\}^T \in \{0,1\}^{n \times d}$: pattern of missing data for the full dataset

■ $c_i = (c_{i1}, \dots, c_{id}) \in \{0,1\}^d$: pattern of missing data for individual $i \in \{1, \dots, n\}$

$$c_{ij} = 1 \Leftrightarrow y_{ij} \text{ is missing}$$

■ $y_i^{\text{obs}}$: the observed variables values for indiv. $i$ (and $y^{\text{obs}} = \{y_1^{\text{obs}}|\dots|y_n^{\text{obs}}\}^T$)

■ $y_i^{\text{mis}}$: the missing variables values for individual $i$ (and $y^{\text{mis}} = \{y_1^{\text{mis}}|\dots|y_n^{\text{mis}}\}^T$)
Typology of the missingness mechanisms

■ Missing completely at random (MCAR):

$$p(c|y; \psi) = p(c; \psi) \quad \forall y$$

■ Missing at random (MAR):

$$p(c|y; \psi) = p(c|y^{\text{obs}}, \psi) \quad \forall y^{\text{mis}}$$

■ Missing not at random (MNAR): the mechanism is not MCAR nor MAR
Ignorable vs. non ignorable model

A missing mechanism is ignorable if likelihoods can be decomposed as

$$L(\theta, \psi; \underbrace{y^{\text{obs}}, c}_{\text{observed data}}) = L(\psi; c|y^{\text{obs}}) \times L(\theta; y^{\text{obs}})$$

Some simple algebra show that this occurs when missing mechanism is not MNAR

Inference of θ

"If the missing mechanism is ignorable then likelihood-based inferences for θ from $L(\theta; y^{\text{obs}})$ will be the same as likelihood based inference for θ from $L(\theta, \psi; y^{\text{obs}}, c)$."

[Little and Rubin, 2002 Section 6.2]

- M(C)AR is ignorable

- MNAR is not ignorable
<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

Mixture model and clustering (Gaussian case)

■ Partition (K clusters): $z = (z_1|...|z_n)^T \in \{0,1\}^{n \times K}$ where

■ $z_i = (z_{i1},...,z_{iK}) \in \{0,1\}^K$

■ $z_{ik} = 1$ if $y_i$ belongs to cluster $k$, $z_{ik} = 0$ otherwise

■ Mixture model: $y_1,...,y_n$ are i.i.d. from the d-variate Gaussian mixture

$$f(y_i; \theta) = \sum_{k=1}^{K} \pi_k f_k(y_i; \alpha_k)$$

■ $\pi_k = p(z_{ik} = 1)$

■ $f_k(.; \alpha_k) = \phi(.; \mu_k, \Sigma_k)$ is the d-variate Gaussian distribution with mean vector $\mu_k$ and covariance matrix $\Sigma_k$

■ $\theta = (\pi_1,..., \pi_K, \alpha_1,..., \alpha_K)$ is the whole mixture parameter

■ Clustering: MAP principle from the mixture output to estimate the partition

And also: algorithm estimation (EM, SEM...) and model selection (BIC, ICL...)
## Outline

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data
<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

Degeneracy genesis: unbounded likelihood

d-variate Gaussian mixture

$$
f(y_i; \theta) = \sum_{k=1}^{K} \pi_k \frac{1}{\left(\frac{(2\pi)^{d/2}}{|\Sigma_k|^{1/2}} \exp\left(-\frac{1}{2}(y_i - \mu_k)^T \Sigma_k^{-1} (y_i - \mu_k)\right)\right)^{\frac{1}{2}}} \phi(y_i; \mu_k, \Sigma_k)
$$

Sampling: $y_1, \dots, y_n \stackrel{i.i.d.}{\sim} p(\cdot; \theta)$ without any missing data ($y^{\text{obs}} = y$)

Likelihood: $\ell(\theta; y) = \ln L(\theta; y) = \sum_{i=1}^{n} \ln f(y_i; \theta)$

$$
\text{particular center } \mu_2 = y_i \quad \Rightarrow \quad \lim_{|\Sigma_2| \to 0} \ell(\theta; y) = +\infty
$$
<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>

EM behaviour: illustration

<figure><img src="image_26.png" /></figure>

■ degeneracy may occur even when starting from large variances

■ convergence can be slow when far from the degenerate limit

■ convergence extremely fast near degeneracy
<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

EM behaviour: results

<figure><img src="image_33.png" /></figure>

$$u_0 = \left[ \frac{1}{p_{i0} k_0}, \{p_{ik_0}\}_{i \neq i_0} \right]$$

degeneracy of component $k_0$ at $y_{i_0}$

⇔

$$\|u_0\| \to 0$$

[Biernacki and Chrétien, 2003]

[Ingrassia and Rocci, 2009]

Proposition 1: Existence of a bassin of attraction

$$\exists \epsilon > 0 \text{ s.t. if } \|u_0\| \le \epsilon \text{ then } \|u_0^+\| = o\|u_0\| \text{ with probability 1.}$$

Proposition 2: Speed towards degeneracy is exponential

$$\exists \epsilon > 0, \alpha > 0 \text{ and } \beta > 0 \text{ s.t. if } \|u_0\| \le \epsilon \text{ then, with probability 1, } \\ \left| \sum_{k_0}^{+} \right| \le \alpha / \left| \sum_{k_0} \right| \cdot \exp \left( -\beta / \left| \sum_{k_0} \right| \right).$$
## Outline

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data
EM behaviour illustration

■ Breast cancer tissue of the UCI database repository: 106 units, 9 variables.

■ 10% of missing data randomly generated

■ K = 4 clusters

Log-likelihood according to the number of iterations

<figure><img src="image_34.png" /></figure>

Decrease of the log-determinant of the degenerated component

<figure><img src="image_35.png" /></figure>
Detail from the illustration

<table><caption>Table : Data belonging to the degenerated component.</caption><thead><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th></tr></thead><tbody><tr><td>1</td><td>211.00</td><td></td><td>0.09</td><td>30.75</td><td>151.98</td><td>4.94</td><td>14.27</td><td>27.24</td><td>217.13</td></tr><tr><td>2</td><td>196.86</td><td>0.02</td><td>0.09</td><td>28.59</td><td>82.06</td><td>2.87</td><td>7.97</td><td>27.66</td><td>200.75</td></tr><tr><td>3</td><td>144.00</td><td>0.12</td><td>0.05</td><td>19.65</td><td>70.43</td><td>3.58</td><td></td><td>7.57</td><td>160.37</td></tr><tr><td>4</td><td>172.52</td><td>0.13</td><td>0.04</td><td></td><td>192.22</td><td>5.12</td><td>19.32</td><td>32.19</td><td>174.93</td></tr><tr><td>5</td><td>121.00</td><td>0.17</td><td>0.09</td><td>24.44</td><td>144.47</td><td>5.91</td><td>22.02</td><td>10.59</td><td>141.77</td></tr><tr><td>6</td><td>223.00</td><td>0.12</td><td>0.08</td><td>33.10</td><td>197.01</td><td>5.95</td><td>30.45</td><td>12.96</td><td>252.48</td></tr><tr><td>7</td><td></td><td>0.17</td><td>0.23</td><td>34.22</td><td>94.35</td><td>2.76</td><td>31.28</td><td>13.88</td><td>180.61</td></tr><tr><td>8</td><td>303.00</td><td>0.06</td><td>0.04</td><td>22.57</td><td></td><td>4.54</td><td>21.83</td><td>5.72</td><td>321.65</td></tr><tr><td>9</td><td>250.00</td><td>0.09</td><td>0.09</td><td>29.64</td><td>180.76</td><td>6.10</td><td>26.14</td><td>13.96</td><td>280.12</td></tr><tr><td>10</td><td>391.00</td><td>0.06</td><td>0.01</td><td>35.78</td><td></td><td>7.41</td><td>22.13</td><td>28.11</td><td>400.99</td></tr><tr><td>11</td><td>176.00</td><td>0.09</td><td>0.08</td><td>20.59</td><td>79.71</td><td></td><td>18.23</td><td>9.58</td><td>191.99</td></tr><tr><td>12</td><td>145.00</td><td></td><td>0.11</td><td>21.22</td><td>82.46</td><td>3.89</td><td>20.30</td><td>6.17</td><td>162.51</td></tr><tr><td>13</td><td>124.13</td><td>0.13</td><td>0.11</td><td>20.59</td><td></td><td></td><td>18.46</td><td>9.12</td><td>134.89</td></tr><tr><td>14</td><td>103.00</td><td>0.16</td><td>0.29</td><td>23.75</td><td>78.26</td><td>3.29</td><td>22.32</td><td>8.12</td><td>124.98</td></tr></tbody></table>

■ Cvg. towards a degenerated component (no plateau of the log-likelihood)

■ Degeneracy relatively slow: log-likelihood linear according to the nb of it.

■ Number of points of the degenerated solution greater than the space dimension d (but the number of complete points lower than d)
<figure><img src="image_36.png" /></figure>

<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>

Intermediate conclusion on missing data

Like the complete data y case

■ Likelihood is unbounded

■ EM can be attracted by degenerate solutions

Unlike the complete data y case

■ Risk to consider a degenerated solution as valid

■ Risk of losing a lot of time in useless iterations

Statisticians should be aware of such dangerous EM behaviour...

... since missing data are more and more frequent
<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

<figure><img src="image_44.png" /></figure>

Understanding degeneracy speed on a toy example

■ Univariate framework, no mixture, only one observed data: $y$

■ Maximum likelihood estimator (Unbounded likelihood!): $\mu = y, \sigma^2 = 0$

■ Suppose equivalently that $n-1$ data are unobserved (unchanged likelihood)

■ Here is one iteration of a (useless) EM algorithm (it. $q$)

$$ \mu^{(q+1)} = \frac{(n-1)\mu^{(q)} + y}{n} \quad \text{and} \quad \sigma^2(q+1) = \frac{(n-1)\sigma^2(q) + (y - \mu^{(q+1)})^2}{n} $$

Linear grow of the log-likelihood (have a look also when $n$ increases!)

$$ \ell(\theta^{(q)}; x) \sim -0.5q \log \frac{n-1}{n} $$

Geometrical convergence rate towards 0 for the variance

$$ \sigma^2(q) \sim \sigma^2(0) \left( \frac{n-1}{n} \right)^q $$
## Outline

### 1 Introduction

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data

### 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration

### 4 Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration

### 5 Concluding remarks
# Outline

## 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration
<figure><img src="image_45.png" /></figure>

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

Full mixed data: conditional independence everywhere<fnref n="1" />

The aim is to combine continuous, categorical, integer data, ordinal, ranking and functional data

$$y_1 = (y_1^{\text{cont}}, y_1^{\text{cat}}, y_1^{\text{int}}, \dots)$$

The proposed solution is to mixed all types by inter-type conditional independence

$$p(y_1; \alpha_k) = p(y_1^{\text{cont}}; \alpha_k^{\text{cont}}) \times p(y_1^{\text{cat}}; \alpha_k^{\text{cat}}) \times p(y_1^{\text{int}}; \alpha_k^{\text{int}}) \times \dots$$

In addition, for symmetry between types, intra-type conditional independence

Only need to define the univariate pdf for each variable type!

* Continuous: Gaussian

* Categorical: multinomial

* Integer: Poisson

* ...

<footnote n="1">¹MixtComp package: https://cran.r-project.org/web/packages/RMixtComp/index.html</footnote>
# Missing data: MAR assumption and estimation

## Assumption on the missingness mechanism

Missing At Randon (MAR): the probability that a variable is missing does not depend on its own value given the observed variables.

## Observed log-likelihood...

$$ \ell(\theta; y^{\text{obs}}) = \sum_{i=1}^{n} \log \left( \sum_{k=1}^{K} \pi_k p(y_i^{\text{obs}}, \alpha_k) \right) = \sum_{i=1}^{n} \ln \left[ \sum_{k=1}^{K} \pi_k \underbrace{\int_{y_i^{\text{mis}}} p(y_i^{\text{obs}}, y_i^{\text{mis}}; \alpha_k) dy_i^{\text{mis}}}_{\text{MAR assumption}} \right] $$
## Outline

3 Impact of MAR data on clustering

■ A model-based MAR approach

■ Biology study illustration
Prostate cancer data (with missing data)<fnref n="3" />

* **Individuals:** 506 patients with prostatic cancer grouped on clinical criteria into two Stages 3 and 4 of the disease

* **Variables:** *d* = 12 pre-trial variates were measured on each patient, composed by eight continuous variables (age, weight, systolic blood pressure, diastolic blood pressure, serum haemoglobin, size of primary tumour, index of tumour stage and histolic grade, serum prostatic acid phosphatase) and four categorical variables with various numbers of levels (performance rating, cardiovascular disease history, electrocardiogram code, bone metastases)

* **Some missing data:** 62 missing values (≈ 1%)

We forget the classes (Stages of the disease) for performing clustering

Questions

* How many clusters?

* Which partition?

<footnote n="3">³Byar DP, Green SB (1980): Bulletin Cancer, Paris 67:477-488</footnote>
<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

Two strategies in competition

■ Strategy "mice<fnref n="4" /> + MixtComp": MixtComp on the dataset completed by mice

> data.imp=mice(data)

> data.comp.mice=complete(data.imp)

■ Strategy "full MixtComp": MixtComp on the observed (no completed) dataset

Partition quality with K = 2

<table><thead><tr><th>Strategy</th><th>mice + MixtComp</th><th>full MixtComp</th></tr></thead><tbody><tr><td>% misclassified</td><td>12.8</td><td>8.1</td></tr></tbody></table>

<footnote n="4">⁴http://cran.r-project.org/web/packages/mice/mice.pdf</footnote>
Choosing K with the ICL criterion

<figure><img src="image_51.png" /></figure>

mice + MixtComp
K = 7

<figure><img src="image_52.png" /></figure>

full MixtComp
K = 2

... may lose some cluster information when imputation before clustering
A by-product zoology of MNAR models

<table><thead><tr><th rowspan="2"></th><th colspan="2">Effect on the variable j</th><th colspan="2">Effect on the class membership k</th><th rowspan="2">Nb parameters<br>Continuous</th></tr><tr><th>Depends on j</th><th>Depends on k</th><th>Depends on j</th><th>Depends on k</th></tr></thead><tbody><tr><td>MNARz<sup>j</sup>y<sup>k</sup></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>2Kd</td></tr><tr><td>MNARy<sup>j</sup></td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>(K + 1)d</td></tr><tr><td>MNARy<sup>k</sup>z</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>K(d + 1)</td></tr><tr><td>MNARyz</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>(K + d)</td></tr><tr><td>MNARy</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>d</td></tr><tr><td>MNARy<sup>k</sup></td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td>Kd</td></tr><tr><td>MNARz</td><td>✗</td><td>✗</td><td>✗</td><td>✓</td><td>K</td></tr><tr><td>MNARz<sup>j</sup></td><td>✗</td><td>✗</td><td>✓</td><td>✓</td><td>Kd</td></tr></tbody></table>

Remarks:

■ MNARz<sup>j</sup>y<sup>k</sup> is the most complex model

■ MNARz, MNARz<sup>j</sup>: the only effect of missingness is on the class membership, $\psi = (\alpha_{11}, \dots, \alpha_{1d}, \dots, \alpha_{K1}, \dots, \alpha_{Kd})^T$, $p(c_{ij} = 1 | y_i, z_{ik} = 1; \psi) = \rho(\alpha_{kj})$

■ MCAR is a specific and simple case
<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>

MNARz analysis: pattern c gives information on partition z!

Draw Bayes error of a MNARz model with two components and 20% of missing data

$\pi_k = 0.5, ||\mu_2 - \mu_1||$ varies, $\Sigma_1 = \Sigma_2 = I, |\alpha_2 - \alpha_1|$ varies

<figure><img src="image_56.png" /></figure>

Both $\mu_k$ and $\alpha_k$ act on the Bayes error
# Reinterpretation of the MNARz and MNARz<sup>j</sup> models as MAR

Commonly used in Machine Learning [Jones, 1996], [Little and Rubin, 2002], [Josse et al., 2019]

Mixture model for $y^{\text{obs}}$ and Bernoulli distribution for $C$

$$ \Leftrightarrow \text{MAR mixture model for } \tilde{y}^{\text{obs}} = (y^{\text{obs}}|c) $$

For example,

$$ y^{\text{obs}} = \begin{pmatrix} ? & 2.6 & 5 \\ \text{blue} & 1.9 & 4 \\ \text{red} & 2.3 & ? \end{pmatrix}, \quad c = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

then $\tilde{y}^{\text{obs}}$ is expressed as

$$ \tilde{y}^{\text{obs}} = \begin{pmatrix} ? & 2.6 & 5 & 1 & 0 & 0 \\ \text{blue} & 1.9 & 4 & 0 & 0 & 0 \\ \text{red} & 2.3 & ? & 0 & 0 & 1 \end{pmatrix}. $$

## Proposition 3: in terms of maximum likelihood

The maximum likelihood estimate associated to the dataset $\tilde{y}^{\text{obs}}$ under MAR model is the one associated to the dataset $y^{\text{obs}}$ under MNARz or MNARz<sup>j</sup> models.
<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>

<figure><img src="image_59.png" /></figure>

# Identifiability

Previous works: [Teicher, 1963] (without NA), [Miao et al., 2016] (for MNAR data)

## Proposition 4

Assume that

1. The marginal mixture $\sum_{k=1}^{K} \pi_k f_k(y_j; \theta_k)$ is identifiable

2. There exists a total ordering $\leq$ of $\mathcal{F}_j \times \mathcal{R}$, for $j \in \{1, \dots, d\}$ fixed, where $\mathcal{F}_j = \{f_{1j}, \dots, f_{Kj}\}$ and $\mathcal{R} = \{\rho_1, \dots, \rho_K\} = \{\rho(\cdot; \psi_1), \dots, \rho(\cdot; \psi_K)\}$. The total ordering is s.t. $\forall k < l$, $F_k = \rho_k f_{kj} \preceq F_l = \rho_l f_{lj}$ implies

$$ \lim_{u \to +\infty} \frac{\rho_l(u)f_{lj}(u)}{\rho_k(u)f_{kj}(u)} = 0 $$

Then the mixture model with one of the MNAR* mechanisms is identifiable up to label swapping

All MNAR models are identifiable (or at least generically identifiable) for probit/logit
<figure><img src="image_60.png" /></figure>

<figure><img src="image_61.png" /></figure>

<figure><img src="image_62.png" /></figure>

Estimation procedure overview

■ Use EM or Stochastic EM (SEM) algorithms

■ MNARz and MNARz<sup>j</sup>: EM and SEM are very simple

■ MNARy*: the SE step requires a within Gibbs loop, sometimes involving itself a Sampling Importance Resampling (SIR)

<table><thead><tr><th></th><th colspan="2">EM</th><th colspan="2">SEM</th></tr><tr><th>MNARz<br>MNARz<sup>j</sup></th><th colspan="2">✓</th><th colspan="2">✓</th></tr><tr><th></th><th>Probit</th><th>Logit</th><th>Probit</th><th>Logit</th></tr></thead><tbody><tr><th>MNARy*</th><td>no closed<br>form</td><td>no closed<br>form,<br>optim. pb</td><td>✓</td><td>require<br>algorithms<br>as SIR<br>(costly)</td></tr></tbody></table>
Model selection

Can select between MCAR and MNAR* with any information criterion (BIC, ICL)

Even if the missing mechanism is ignorable for MCAR...

... need to model c to compare a MCAR and a MNAR model

CAUTION

- It is just a selection between several proposed MNAR models

- It is not deciding if missingness procedure is “genererically” MNAR or not
<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

<figure><img src="image_65.png" /></figure>

Hospital data description

■ Number of patients: $n = 5146$

■ Number of features: $d = 7$

■ Age

■ Size

■ Weight

■ Cardiac frequency

■ Hemoglobin concentration

■ Temperature

■ Minimum Diastolic and Systolic Blood Pressure

■ Percentage of missing data: 6.4%

Doctors are convinced that their missing data are MNAR
<figure><img src="image_66.png" /></figure>

<figure><img src="image_67.png" /></figure>

<figure><img src="image_68.png" /></figure>

ICL comparison

<figure><img src="image_69.png" /></figure>

■ MCAR, MNARy and MNARz are equivalent until $K = 3$

■ MNARz and MNARyz clearly indicate presence of an additional cluster ($K = 4$)

It seems to be an illustration of the effect of c through MNARz and MNARyz
## Outline

### 1 Introduction

### 2 Impact of MAR data on the EM algorithm

- Gaussian mixture degeneracy *without* missing data

- Gaussian mixture degeneracy *with* missing data

### 3 Impact of MAR data on clustering

- A model-based MAR approach

- Biology study illustration

### 4 Impact of MNAR data on clustering

- A model-based MNAR clustering approach

- Medical study illustration

### 5 Concluding remarks
<figure><img src="image_70.png" /></figure>

<figure><img src="image_71.png" /></figure>

<figure><img src="image_72.png" /></figure>

To conclude

Summary

■ Statisticians should properly consider the potential missing data impact both
from an algorithmic and from a modeling point of view

■ EM: be careful about degeneracy which seems to exacerbated/masqued

■ MAR: the minimal approach to be used by statisticians...

■ MNAR: interest of the simple but meaningful model MNARz, link with usual methods

Ongoing works

■ EM: propose mechanism to identify/discard degenerate runs

■ MNAR: extend to categorical, count and mixed data

Thanks!