Examining the robustness of a model selection
procedure in the binary latent block model
through a language placement test data set

Vincent Brault, Frédérique Letué, Marie-José Martinez

Univ. Grenoble Alpes, CNRS, Grenoble INP<fnref n="1" />, LJK, 38000 Grenoble, France

**Abstract**

When entering French university, the students’ foreign language level is as-
sessed through a placement test. In this work, we model the placement test
results using binary latent block models which allow to simultaneously form
homogeneous groups of students and of items. However, a major difficulty in
latent block models is to select correctly the number of groups of rows and
the number of groups of columns. The first purpose of this paper is to tune
the number of initializations needed to limit the initial values problem in the
estimation algorithm in order to propose a model selection procedure in the
placement test context. Computational studies based on simulated data sets
and on two placement test data sets are investigated. The second purpose
is to investigate the robustness of the proposed model selection procedure in
terms of stability of the students groups when the number of students varies.

***Keywords:*** Latent block model, Model selection, Robustness, Placement
test data.

# 1. Introduction

When entering French university, students may have different foreign lan-
guage levels. They need to be evaluated before being directed in an adequate
foreign language class. To assess their level, universities use different place-
ment tests. The SELF test developed in Grenoble is one of the most used
([1, 2]). At the end of this test, each student get an aggregated score which

<footnote n="1">¹Institute of Engineering Univ. Grenoble Alpes</footnote>
corresponds to the course level where he/she has to register in. He/she also gets a mark for each of the three evaluated skills (oral comprehension, written comprehension and written expression). Moreover, the test creators need to check the relevance of the questions, in particular whether they are discriminating or not. It may be useful to form groups of items more or less difficult per skill.

Results of such a placement test can be displayed as a matrix where each row corresponds to one student and each column to one item. Element $(i, j)$ of the matrix equals 1 if student $i$ answers correctly question $j$, and 0 otherwise. Latent block models ([3]) aim to simultaneously achieve a clustering of the rows and the columns and thus turn out to be particularly useful to form homogeneous groups of students and of items in the placement test context.

Classical algorithms to estimate parameters in latent block models, namely Variational or Stochastic Expectation Maximization ([3, 4, 5]), are extensions of the EM-algorithm ([6]). Although these algorithms can give satisfactory estimates, they appear to be quite sensitive to starting values and have a marked tendency to provide empty clusters. To overcome these limitations, Keribin et al. [5] proposed several algorithms through Bayesian inference using Gibbs sampling.

A major difficulty in latent block models is to select correctly the number of groups of rows and the number of groups of columns. For this purpose, penalized likelihood criteria such as Akaike Information Criterion (AIC, [7, 8]) or Bayesian Information Criterion (BIC, [9]) are not directly usable since computing the maximized likelihood is not possible. Another widely used criterion is the Integrated Completed Likelihood criterion (ICL) defined by Biernacki et al. [10]. This criterion has been extended to the latent block model for binary data in [11] and for categorical data in [5].

The first purpose of this paper is to tune the number of initializations needed to limit the initial values problem in the estimation algorithm in order to propose a model selection procedure for latent block models on binary data in the context of placement tests. Two computational studies based on simulated data sets and on the two placement test data sets described in Section 2 are investigated. The second purpose of this paper is to investigate the robustness of the proposed model selection procedure. The robustness is here assessed in terms of the stability of the students groups when the number of students varies.
The paper is organized as follows. The two placement test data sets used in this paper are described in the next section. In Section 3, we recall the statistical model, the notations and the estimation algorithm and we present the model selection procedure based on ICL criterion considered in this work. Section 4 deals with the effect of initialization strategies on the model selection procedure. The robustness of the proposed procedure is investigated in Section 5. Finally, a conclusion presenting a perspective for future work ends this paper.

## 2. Data sets

The data sets considered in this paper have been obtained from the SELF placement test developed at Université Grenoble Alpes in 6 languages (English, French as a Foreign Language, Italian, Japanese, Mandarin Chinese and Spanish) and used at a number of partner universities in France (see [1]). The SELF test is a semi-adaptive multi-stage test. The first stage (the initial testlet) is common to all test takers, but the items in the second stage depend on test takers’ results in the first. Results in the second stage are used to refine the estimation of learners’ level and arrive at placement results expressed in Common European Framework of Reference (CEFR) levels that are as reliable as possible. In this paper, the results of a Japanese and an English SELF placement tests are considered. The Japanese (resp. English) SELF test has been taken by 137 (resp. 228) students entering Université Grenoble Alpes in 2019. We focus here only on the first stage composed with 33 (resp. 36) items common to all test takers. These items are labelled with three skills: oral comprehension, written comprehension and written expression.

Figure 1 displays the English SELF placement test results for the 228 students (lines) and the 36 items (columns). Each element of the matrix is colored in white when the student answers correctly the question, and in black otherwise.

## 3. Statistical framework

In this section, the statistical model, the notations and the estimation algorithm are first described. Then, the model selection procedure based on ICL criterion is presented.
<figure><img src="image_1.png" /><figcaption>Figure 1: Results of the English SELF placement test (on the left) for 228 students (lines) and 36 items (columns) and the Japanese SELF placement test (on the right) for 137 students (lines) and 33 items (columns). A white cell corresponds to a correct answer.</figcaption></figure>
### 3.1. Statistical model and notations

Let $n$ be the number of students, $g$ the number of students groups, $q$ the number of items and $m$ the number of items groups. We define $Z_i, i = 1 \cdots n$, as the independent random variables modelling the students group which student $i$ belongs to. The $Z_i, i = 1 \cdots n$ variables follow the multinomial $M(1; \pi_1, \dots, \pi_g)$ distribution. We also define $W_j, j = 1 \cdots q$, as the independent random variables modelling the items group which item $j$ belongs to. The $W_j, j = 1 \cdots q$ variables follow the multinomial $M(1; \rho_1, \dots, \rho_m)$ distribution.

Responses are assumed to be independent. Given that student $i$ belongs to group $k$ and item $j$ belongs to group $l$, the response of student $i$ to item $j$ denoted by $Y_{ij}$ follows the Bernoulli distribution with parameter $\alpha_{kl}$:

$$P(Y_{ij} = 1 | Z_i = k, W_j = l) = \alpha_{kl}.$$

### 3.2. Estimation algorithm

A classical method to estimate parameters in latent variables models is the EM-algorithm. [3, 4] show that the EM-algorithm cannot directly be used in practice. Indeed, it would require to calculate a sum over all possible couples $(z_i, w_j), 1 \le i \le n, 1 \le j \le q$, which would have a too high computational cost. To overcome this problem, [4] propose a variational approximation of the EM-algorithm (VEM) based on a decomposition of the log-likelihood as the sum of the free energy and the Kullback-Leibler divergence ([12]). Since the Kullback-Leibler divergence is expected to be small around the log-likelihood maximum, they propose to only maximise the free energy. In [13], the authors show that the free energy maximum estimator is consistent.

As pointed by Govaert and Nadif [4] and Keribin et al. [5], the estimated parameters obtained by the VEM-algorithm highly depend on its initial values. Moreover, Keribin et al. [5] show that this algorithm tends to provide empty groups. To overcome these problems, they propose a V-Bayes algorithm to avoid empty groups, combined with a Gibbs sampler to limit the initial values problem. In this bayesian approach, proper and independent informative prior distributions are considered for the parameters. The mixing proportions $\pi = (\pi_1, \dots, \pi_g)$ and $\rho = (\rho_1, \dots, \rho_m)$ are assumed to be Dirichlet-distributed with parameters $(a, \dots, a)$. The parameters $\alpha_{11}, \dots, \alpha_{gm}$ are assumed to be Beta-distributed with parameter $(b, b)$.
In Section 4, hyperparameters a and b in the estimation algorithm are chosen to be equal to 4 and 1 respectively, as advised by Keribin et al. [5]. In this paper, we use the combined V-Bayes Gibbs algorithm to get estimators $\hat{\pi}$, $\hat{\rho}$, $\hat{\alpha}$ of the parameters for a given number of students groups g and a given number of items groups m. Furthermore, in order to limit the initial values problem, this algorithm is run T times and we finally keep the estimators $\hat{\pi}$, $\hat{\rho}$, $\hat{\alpha}$ which maximize the free energy over the T runs.

### 3.3. Model selection procedure and the ICL criterion

In this subsection, we investigate the ICL model selection criterion in order to propose a model selection procedure.

The ICL criterion has been defined in [10] as the logarithm of the integrated completed likelihood in a mixture model context. It has been extended to the latent block model for binary data in [11] and for categorical data in [5].

Using the conditional independence of the z's and the w's conditionally to $\pi$, $\rho$ and $\alpha$, the conjugate properties of the prior Dirichlet distributions and the conditional independence of the $y_{ij}$ given the latent variables z and w, the ICL criterion can be written as

$$ 
\begin{aligned}
ICL_{(z,w)}(g, m) = & \log \Gamma(ga) + \log \Gamma(ma) - (m + g) \log \Gamma(a) \\
& +mg(\log \Gamma(2b) - 2 \log \Gamma(b)) - \log \Gamma(n + ga) - \log \Gamma(q + ma) \\
& + \sum_k \log \Gamma(z_{.k} + a) + \sum_l \log \Gamma(w_{.l} + a) \\
& + \sum_{k,l} \left[ \log \Gamma(N_{kl}^1 + b) + \log \Gamma(N_{kl}^0 + b) - \log \Gamma(z_{.k}w_{.l} + 2b) \right]
\end{aligned}
$$

where

$$ z_{.k} = \sum_{i} 1_{\{z_i=k\}}, $$

$$ w_{.l} = \sum_{j} 1_{\{w_j=l\}}, $$

$$ N_{kl}^1 = \sum_{i,j} y_{ij} 1_{\{z_i=k, w_j=l\}}, $$

$$ N_{kl}^0 = \sum_{i,j} (1 - y_{ij}) 1_{\{z_i=k, w_j=l\}}. $$

Details can be found in [5].
Given $(g, m)$, the maximization of $ICL_{(z,w)}(g, m)$ over all partitions $(z, w)$ would require a too high computational cost. In their paper, Biernacki et al. biernacki2000 proposed to use $ICL_{(\hat{z}, \hat{w})}(g, m)$ where $(\hat{z}, \hat{w})$ is the maximizing partition obtained with a maximum a posteriori (MAP) rule after the last V-Bayes step.

In order to select the numbers of students groups $g$ and of items groups $m$, we calculate every criterion $ICL_{(\hat{z},\hat{w})}(g, m)$ where $(g, m)$ runs on a given grid, and we finally define $(\hat{g}, \hat{m})$ as the pair that maximizes these criteria.

# 4. Effect of initialization strategies on the model selection procedure

The objective of this section is to tune the parameter $T$, namely the number of initializations needed to limit the initial values problem in the estimation algorithm. For that purpose, we investigate two computational studies. The first one is based on simulated data sets, and the second one is based on the two real data sets described in Section 2.

## 4.1. Simulation study

### 4.1.1. Simulation plan and indicators

In this simulation study, we simulate $L = 100$ data sets from the model described in Section 3.1. Mimicking the Japanese SELF placement test data set, we set $n = 137$ and $q = 33$. We also set $g = 3$, $m = 4$, $\pi_1 = \pi_2 = \pi_3 = \frac{1}{3}$ and $\rho_1 = \rho_2 = \rho_3 = \rho_4 = \frac{1}{4}$. Finally, we define parameter $\alpha_{kl}$ equal to $\varepsilon$ if $k \ge l$ and $1 - \varepsilon$ if $k < l$ with $\varepsilon \in \{0.05, 0.15, 0.2, 0.25, 0.3\}$. Note that parameter $\varepsilon$ can be considered as an indicator of the estimation difficulty.

In order to tune parameter $T$, we run the following algorithm:

Table 1 displays the distribution of parameter $T$ for each value of $\varepsilon$.

### 4.1.2. Results

Results displayed in Table 1 show that for small values of $\varepsilon$ ($\varepsilon \le 0.25$), $T = 1$ is enough to select the simulated model ($g = 3$ and $m = 4$) most of the time (96% for $\varepsilon = 0.05$, 99% for $\varepsilon = 0.15$, 98% for $\varepsilon = 0.2$ and 79% for $\varepsilon = 0.25$). That means that for simple cases running the estimation algorithm once per pair $(g, m)$ is usually sufficient.
for $\varepsilon \in \{0.05, 0.15, 0.2, 0.25, 0.3\}$ do
    for $L$ from 1 to 100 do
        simulate a data set
        $T = 0$
        $(\hat{g}, \hat{m}) = (1, 1)$
        while $(\hat{g}, \hat{m}) \neq (3, 4)$ do
            $T = T + 1$
            for each pair $(g, m)$, with $g$ and $m$ varying from 1 to 7 do
                1. calculate the estimators $(\hat{\pi}, \hat{\rho}, \hat{\alpha})$;
                2. calculate the maximizing a posteriori partition $(\hat{z}, \hat{w})$;
                3. calculate the associated ICL value;
            end
            select the pair $(\hat{g}, \hat{m})$ that maximizes the ICL criterion.
        end
    end
end

<table><caption>Table 1: Distribution of $T$ (columns) in function of the value $\varepsilon$ (rows).</caption><thead><tr><th></th><th>1</th><th>2</th><th>14</th><th>≥ 50</th></tr></thead><tbody><tr><td>$\varepsilon = 0.05$</td><td>96</td><td>4</td><td>0</td><td>0</td></tr><tr><td>$\varepsilon = 0.15$</td><td>99</td><td>1</td><td>0</td><td>0</td></tr><tr><td>$\varepsilon = 0.2$</td><td>98</td><td>0</td><td>0</td><td>2</td></tr><tr><td>$\varepsilon = 0.25$</td><td>79</td><td>0</td><td>1</td><td>20</td></tr><tr><td>$\varepsilon = 0.3$</td><td>24</td><td>0</td><td>0</td><td>76</td></tr></tbody></table>

For $\varepsilon = 0.3$, we can see from Table 1 that the distribution of parameter $T$ is shifted to higher values. That means that for more difficult cases, we need to run the estimation algorithm a high number of times per pair $(g, m)$ to retrieve the simulated model.

## 4.2. Study based on real data sets

In this subsection, we evaluate the performance of our procedure from the two data sets described in Section 2. For that purpose, we proceed in two steps.

First, since there is no "true" model in a real dataset context, we run the
following algorithm a great number of times $K = 170 000$ to get a reference model :

for $k = 1$ to $K$ do
  for each pair $(g, m)$, with $g$ and $m$ varying from 1 to 7 do
    1. calculate the estimators $(\hat{\pi}, \hat{\rho}, \hat{\alpha})$;
    2. calculate the maximizing a posteriori partition $(\hat{z}, \hat{w})$;
    3. calculate the associated ICL value;
  end
  $(\hat{g}, \hat{m})_k$ is the pair $(\hat{g}, \hat{m})$ that maximizes the ICL criterion.
end
$(g, m)_{ref}$ is the pair $(\hat{g}, \hat{m})_k$ that maximizes the ICL criterion over the $K$ pairs.

Secondly, we consider the number of times this algorithm finds the reference model over the $K$ runs and we study the inter-arrival times of the reference model along the $K$ runs.

### 4.2.1. The Japanese SELF placement test data set

For the Japanese SELF dataset, the obtained reference model is (3, 4). Over the $K = 170 000$ runs, the reference model has been obtained 170 000 times. That means that the inter-arrival times are all equal to 1. This dataset can be considered as an "easy" case, for which only one initialization is needed.

Table 2 gives one realization of the estimation of each parameter and Figure 2 displays a summarized representation of the parameters.

<table><caption>Table 2: A realization of the estimation of $\rho$ (top), $\pi$ (left) and $\alpha$ (bottom right) for the Japanese data set.</caption><tr><td></td><td>0.291</td><td>0.264</td><td>0.299</td><td>0.147</td></tr><tr><td>0.329</td><td>0.845</td><td>0.952</td><td>0.995</td><td>0.987</td></tr><tr><td>0.342</td><td>0.414</td><td>0.736</td><td>0.919</td><td>0.916</td></tr><tr><td>0.329</td><td>0.259</td><td>0.37</td><td>0.388</td><td>0.738</td></tr></table>
<figure><img src="image_2.png" /><figcaption>Figure 2: Data representation after the rows and columns have been ordered by classes (left) and estimated model representation (right) for the Japanese SELF test data set. The closer the $\hat{\alpha}_{kl}$ value is to 1, the whiter the block.</figcaption></figure>

### 4.2.2. The English SELF placement test data set

For the English SELF test data set, the obtained reference model is (4, 5). This reference model has been obtained only 16 times over the $K = 170 000$ runs. Thus, we observe 16 inter-arrival times. Table 3 displays the distribution of these 16 values. They vary between 700 and 36 345, with a median value equal to 6595.5. This means that we should run at least 6600 initializations to obtain the reference model with probability 1/2.

Table 4 gives one realization of the estimation of each parameter and Figure 3 displays a summarized representation of the parameters.

## 4.3. Discussion

The simulation study carried out previously shows that the results can be unstable depending on the difficulty of the considered case. That is why, in practice, we encourage to first run the model selection procedure with
<table><caption>Table 3: Distribution of the inter-arrival times of the reference model (4, 5) for the English SELF dataset.</caption><tr><td>Max.</td><td>36345</td></tr><tr><td>3rd Qu.</td><td>13398.5</td></tr><tr><td>Mean</td><td>10534.125</td></tr><tr><td>Median</td><td>6595.5</td></tr><tr><td>1st Qu.</td><td>4533.75</td></tr><tr><td>Min.</td><td>700</td></tr></table>

<table><caption>Table 4: A realization of the estimation of $\rho$ (on top), $\pi$ (on left) and $\alpha$ (on bottom right) for the English SELF test data set.</caption><tr><td></td><td>0.137</td><td>0.235</td><td>0.256</td><td>0.117</td><td>0.254</td></tr><tr><td>0.145</td><td>0.641</td><td>0.779</td><td>0.905</td><td>0.784</td><td>0.958</td></tr><tr><td>0.106</td><td>0.0303</td><td>0.664</td><td>0.931</td><td>0.766</td><td>0.922</td></tr><tr><td>0.317</td><td>0.257</td><td>0.476</td><td>0.637</td><td>0.724</td><td>0.816</td></tr><tr><td>0.432</td><td>0.0334</td><td>0.295</td><td>0.424</td><td>0.556</td><td>0.538</td></tr></table>

only one initialization in the estimation algorithm ($T = 1$) and then examine the estimated values $\hat{\alpha}_{kl}$ of the parameters $\alpha_{kl}$ in the selected model. If the $\hat{\alpha}_{kl}$ matrix lines and columns have different profiles, we can conclude that the case is quite simple and there is no need to increase the number of initializations in the estimation algorithm. This is for instance the case in the Japanese SELF placement test data set (see Figure 2). On the contrary, when the profiles are similar, it would be relevant to increase the number of initializations in order to stabilize the procedure. As an illustration, one can see in Figure 3 that columns 3 and 5 show quite similar profiles which may explain the selection model difficulties. To determine the relevant number of initializations, a possibility could be to run a simulation study mimicking the real data set from the estimated parameters in order to examine the results stability with respect to $T$.
<figure><img src="image_3.png" /><figcaption>Figure 3: Data representation after the rows and columns have been ordered by classes (left) and estimated model representation (right) for the English SELF test data set. The closer the $\hat{\alpha}_{kl}$ value is to 1, the whiter the block.</figcaption></figure>

5. Robustness

This section is devoted to a robustness study of the proposed model selection procedure in the following senses:
• the number of students groups with respect to the sample size,

• the belonging of two given students to a same group with respect to the sample size.

5.1. *Sampling plan and indicators*

To explore the robustness of the proposed model selection procedure, we simulate $L = 100$ data sets from the simulation plan described in Section 4.1.1 with $\varepsilon = 0.15, 0.2$ and $0.25$. For $L = 1$ to $100$, we run the algorithm once and we check that $(\hat{g}, \hat{m}) = (3, 4)$. If this is not the case, we simulate another data set. While running the algorithm, we get the estimated students groups proportions $\hat{\pi}_1, \hat{\pi}_2$ and $\hat{\pi}_3$.

For a given students sample size ($n = 20, 40, 60, ..., 120$), we draw 10 students samples from the 137 students respecting the $\hat{\pi}_1, \hat{\pi}_2$ and $\hat{\pi}_3$ proportions and we apply our procedure to these 10 samples. We display in Tables 6, 7 and 8, for each value of $\varepsilon$, the distribution of the $(\hat{g}, \hat{m})$ pairs selected by the proposed model selection procedure with respect to $n$ over the $100 \times 10$ samples.

In a second step, we compare the *n*-students partition with the initial 137-students partition. For that purpose, following [14] extending [15],

• when the selected number of students groups is equal to $g = 3$, we draw the contingency table of the students belonging groups in the reference model and in the selected model. Students on the diagonal are defined as well classified whereas the students out of the diagonal are defined as misclassified. As illustrated in Table 5, we consider all possible labels switching and we keep the one that gives the smallest misclassified students number.

• when the selected number of students groups is greater than $g = 3$, we consider all possible groups unions of the selected model in order to get only 3 groups and we keep the group union that gives the smaller misclassified students number,

• when the selected number of students groups is smaller than $g = 3$, we consider all possible groups unions of the reference model and we proceed as previously.
<table><caption>Table 5: Example of label switching ([14]). In the right table, the number of misclassified students is equal to 13. By switching groups $\hat{G}2$ and $\hat{G}3$, we obtained a lower number of misclassified students.</caption><thead><tr><th>Ref</th><th>G1</th><th>G2</th><th>G3</th><th>Total</th></tr></thead><tbody><tr><th>G1</th><td>6</td><td>1</td><td>1</td><td>8</td></tr><tr><th>G2</th><td>0</td><td>1</td><td>6</td><td>7</td></tr><tr><th>G3</th><td>0</td><td>5</td><td>0</td><td>5</td></tr><tr><th>Total</th><td>6</td><td>7</td><td>7</td><td>20</td></tr></tbody></table><table><thead><tr><th>Ref</th><th>G1</th><th>G3</th><th>G2</th><th>Total</th></tr></thead><tbody><tr><th>G1</th><td>6</td><td>1</td><td>1</td><td>8</td></tr><tr><th>G2</th><td>0</td><td>6</td><td>1</td><td>7</td></tr><tr><th>G3</th><td>0</td><td>0</td><td>5</td><td>5</td></tr><tr><th>Total</th><td>6</td><td>7</td><td>7</td><td>20</td></tr></tbody></table>

Tables 6, 7 and 8 display the distribution of $(\hat{g}, \hat{m})$ pairs selected by the proposed procedure with respect to $\varepsilon$ and $n$. We can observe that, for $\varepsilon = 0.15$, the distribution of $(\hat{g}, \hat{m})$ is very well concentrated on the reference pair (3, 4), even when the number of students $n$ is small. As $\varepsilon$ increases, the distribution is more scattered, mainly for small $n$ values. Nevertheless, for $\varepsilon = 0.25$, when $n$ increases, we retrieve a concentrated distribution around the reference pair.

In Figures 4, 5 and 6, we focus on the numbers of students groups. The misclassified students rate distribution is displayed with respect to the number of students $n$ and the selected number $\hat{g}$ of students groups. Note that this rate cannot be greater than $(g-1)/g$ ([14]). When $\hat{g} = 3$ (the reference number of students groups, blue boxplots), the misclassified students rate decreases when $n$ increases. When $\hat{g} = 2$ (green boxplots), the rate of misclassified students does not tend to decrease but this value of $\hat{g}$ is less and less selected. Note that values 1 and 4 are very rarely selected, whatever the values of $n$ and $\varepsilon$.
<table><caption>Table 6: Distribution of the $(\hat{g}, \hat{m})$ pairs selected by the proposed procedure with respect to $n$ for $\varepsilon = 0.15$. The boxed data corresponds to the reference pair.</caption><thead><tr><th colspan="5"></th><th colspan="5">ε = 0.15</th></tr></thead><tbody><tr><th colspan="5">n = 20</th><th colspan="5">n = 80</th></tr><tr><th>m</th><th>3</th><th>4</th><th>5</th><th>6</th><th>Total</th><th>m</th><th>3</th><th>4</th><th>5</th><th>6</th><th>Total</th></tr><tr><th>g</th><th></th><th></th><th></th><th></th><th></th><th>g</th><th></th><th></th><th></th><th></th><th></th></tr><tr><th>2</th><td>190</td><td>13</td><td>0</td><td>0</td><td>203</td><th>2</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th>3</th><td>45</td><td>740</td><td>2</td><td>0</td><td>787</td><th>3</th><td>990</td><td>10</td><td>0</td><td>0</td><td>1000</td></tr><tr><th>4</th><td>9</td><td>1</td><td>0</td><td>0</td><td>10</td><th>4</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th colspan="5">n = 40</th><th colspan="5">n = 100</th></tr><tr><th>m</th><th>3</th><th>4</th><th>5</th><th>6</th><th>Total</th><th>m</th><th>3</th><th>4</th><th>5</th><th>6</th><th>Total</th></tr><tr><th>g</th><th></th><th></th><th></th><th></th><th></th><th>g</th><th></th><th></th><th></th><th></th><th></th></tr><tr><th>2</th><td>18</td><td>10</td><td>0</td><td>0</td><td>28</td><th>2</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th>3</th><td>0</td><td>953</td><td>10</td><td>0</td><td>963</td><th>3</th><td>998</td><td>1</td><td>0</td><td>0</td><td>999</td></tr><tr><th>4</th><td>0</td><td>9</td><td>0</td><td>0</td><td>9</td><th>4</th><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th colspan="5">n = 60</th><th colspan="5">n = 120</th></tr><tr><th>m</th><th>3</th><th>4</th><th>5</th><th>6</th><th>Total</th><th>m</th><th>3</th><th>4</th><th>5</th><th>6</th><th>Total</th></tr><tr><th>g</th><th></th><th></th><th></th><th></th><th></th><th>g</th><th></th><th></th><th></th><th></th><th></th></tr><tr><th>2</th><td>0</td><td>10</td><td>0</td><td>0</td><td>10</td><th>2</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th>3</th><td>0</td><td>979</td><td>11</td><td>0</td><td>990</td><th>3</th><td>990</td><td>1</td><td>9</td><td>1000</td><td>0</td></tr><tr><th>4</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><th>4</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr></tbody></table>
<table><caption>Table 7: Distribution of the $(\hat{g}, \hat{m})$ pairs selected by the proposed procedure with respect to $n$ for $\epsilon = 0.20$. The boxed data corresponds to the reference pair.</caption><caption>ε = 0.20</caption><tbody><tr><th colspan="6" style="text-align:center;">n = 20</th><th colspan="6" style="text-align:center;">n = 80</th></tr><tr><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr><tr><th style="text-align:center;">g</th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th></tr><tr><th style="text-align:center;">1</th><td>28</td><td>1</td><td>0</td><td>0</td><td>29</td><th style="text-align:center;">1</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th style="text-align:center;">2</th><td>52</td><td>485</td><td>67</td><td>0</td><td>604</td><th style="text-align:center;">2</th><td>0</td><td>0</td><td>30</td><td>0</td><td>30</td></tr><tr><th style="text-align:center;">3</th><td>1</td><td>108</td><td>239</td><td>0</td><td>348</td><th style="text-align:center;">3</th><td>0</td><td>0</td><td>960</td><td>10</td><td>970</td></tr><tr><th style="text-align:center;">4</th><td>0</td><td>19</td><td>0</td><td>0</td><td>19</td><th style="text-align:center;">4</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th colspan="6" style="text-align:center;">n = 40</th><th colspan="6" style="text-align:center;">n = 100</th></tr><tr><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr><tr><th style="text-align:center;">g</th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th></tr><tr><th style="text-align:center;">1</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><th style="text-align:center;">1</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th style="text-align:center;">2</th><td>1</td><td>119</td><td>79</td><td>0</td><td>199</td><th style="text-align:center;">2</th><td>0</td><td>0</td><td>1</td><td>0</td><td>1</td></tr><tr><th style="text-align:center;">3</th><td>0</td><td>9</td><td>782</td><td>9</td><td>800</td><th style="text-align:center;">3</th><td>0</td><td>0</td><td>980</td><td>19</td><td>999</td></tr><tr><th style="text-align:center;">4</th><td>0</td><td>0</td><td>1</td><td>0</td><td>1</td><th style="text-align:center;">4</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th colspan="6" style="text-align:center;">n = 60</th><th colspan="6" style="text-align:center;">n = 120</th></tr><tr><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr><tr><th style="text-align:center;">g</th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th></tr><tr><th style="text-align:center;">1</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><th style="text-align:center;">1</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th style="text-align:center;">2</th><td>0</td><td>11</td><td>49</td><td>9</td><td>69</td><th style="text-align:center;">2</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td></tr><tr><th style="text-align:center;">3</th><td>0</td><td>0</td><td>928</td><td>3</td><td>931</td><th style="text-align:center;">3</th><td>0</td><td>0</td><td>989</td><td>10</td><td>999</td></tr><tr><th style="text-align:center;">4</th><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><th style="text-align:center;">4</th><td>0</td><td>0</td><td>1</td><td>0</td><td>1</td></tr></tbody></table>
<table><caption>Table 8: Distribution of the $(\hat{g}, \hat{m})$ pairs selected by the proposed procedure with respect to $n$ for $\varepsilon = 0.25$. The boxed data corresponds to the reference pair.</caption><thead><tr><th colspan="6" style="text-align:center;">ε = 0.25</th></tr><tr><th colspan="6" style="text-align:center;">n = 20</th></tr><tr><th style="text-align:center;">g</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr></thead><tbody><tr><td style="text-align:center;">1</td><td></td><td style="text-align:center;">228</td><td style="text-align:center;">9</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">237</td></tr><tr><td style="text-align:center;">2</td><td></td><td style="text-align:center;">185</td><td style="text-align:center;">440</td><td style="text-align:center;">10</td><td style="text-align:center;">0</td><td style="text-align:center;">635</td></tr><tr><td style="text-align:center;">3</td><td></td><td style="text-align:center;">10</td><td style="text-align:center;">104</td><td style="text-align:center;">12</td><td style="text-align:center;">0</td><td style="text-align:center;">126</td></tr><tr><td style="text-align:center;">4</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">2</td><td style="text-align:center;">0</td><td style="text-align:center;">2</td></tr></tbody></table><table><thead><tr><th colspan="6" style="text-align:center;">n = 40</th></tr><tr><th style="text-align:center;">g</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr></thead><tbody><tr><td style="text-align:center;">1</td><td></td><td style="text-align:center;">19</td><td style="text-align:center;">11</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">30</td></tr><tr><td style="text-align:center;">2</td><td></td><td style="text-align:center;">10</td><td style="text-align:center;">579</td><td style="text-align:center;">97</td><td style="text-align:center;">0</td><td style="text-align:center;">686</td></tr><tr><td style="text-align:center;">3</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">21</td><td style="text-align:center;">245</td><td style="text-align:center;">18</td><td style="text-align:center;">284</td></tr><tr><td style="text-align:center;">4</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr></tbody></table><table><thead><tr><th colspan="6" style="text-align:center;">n = 60</th></tr><tr><th style="text-align:center;">g</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr></thead><tbody><tr><td style="text-align:center;">1</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr><tr><td style="text-align:center;">2</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">217</td><td style="text-align:center;">109</td><td style="text-align:center;">0</td><td style="text-align:center;">326</td></tr><tr><td style="text-align:center;">3</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">606</td><td style="text-align:center;">57</td><td style="text-align:center;">663</td></tr><tr><td style="text-align:center;">4</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">11</td><td style="text-align:center;">0</td><td style="text-align:center;">11</td></tr></tbody></table><table><thead><tr><th colspan="6" style="text-align:center;">n = 80</th></tr><tr><th style="text-align:center;">g</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr></thead><tbody><tr><td style="text-align:center;">1</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr><tr><td style="text-align:center;">2</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">142</td><td style="text-align:center;">137</td><td style="text-align:center;">9</td><td style="text-align:center;">288</td></tr><tr><td style="text-align:center;">3</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">674</td><td style="text-align:center;">38</td><td style="text-align:center;">712</td></tr><tr><td style="text-align:center;">4</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr></tbody></table><table><thead><tr><th colspan="6" style="text-align:center;">n = 100</th></tr><tr><th style="text-align:center;">g</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr></thead><tbody><tr><td style="text-align:center;">1</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr><tr><td style="text-align:center;">2</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">9</td><td style="text-align:center;">80</td><td style="text-align:center;">0</td><td style="text-align:center;">89</td></tr><tr><td style="text-align:center;">3</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">891</td><td style="text-align:center;">20</td><td style="text-align:center;">911</td></tr><tr><td style="text-align:center;">4</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr></tbody></table><table><thead><tr><th colspan="6" style="text-align:center;">n = 120</th></tr><tr><th style="text-align:center;">g</th><th style="text-align:center;">m</th><th style="text-align:center;">2</th><th style="text-align:center;">3</th><th style="text-align:center;">4</th><th style="text-align:center;">5</th><th style="text-align:center;">Total</th></tr></thead><tbody><tr><td style="text-align:center;">1</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr><tr><td style="text-align:center;">2</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">58</td><td style="text-align:center;">9</td><td style="text-align:center;">67</td></tr><tr><td style="text-align:center;">3</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">924</td><td style="text-align:center;">9</td><td style="text-align:center;">933</td></tr><tr><td style="text-align:center;">4</td><td></td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td><td style="text-align:center;">0</td></tr></tbody></table></table>
<figure><img src="image_4.png" /><figcaption>Figure 4: Misclassified students rate distribution with respect to the number *n* of students and the selected number $\hat{g}$ of students groups for $\varepsilon = 0.15$. Each color corresponds to a given selected number of students groups ($\hat{g} = 2, 3, 4$). The sample size *n* is followed by the numbers of samples for each boxplot, when these numbers are greater or equal to 10. For example, the first three boxplots correspond to *n* = 20. The left (resp. middle and right) one is drawn from the 203 (resp. 787 and 10) samples from which $\hat{g} = 2$ (resp. 3 and 4) groups have been selected.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Figure 5: Misclassified students rate distribution with respect to the number *n* of students and the selected number $\hat{g}$ of students groups for $\varepsilon = 0.2$. Each color corresponds to a given selected number of students groups ($\hat{g} = 1, 2, 3, 4$). The sample size *n* is followed by the numbers of samples for each boxplot, when these numbers are greater or equal to 10. For example, the first four boxplots correspond to *n* = 20. The red (resp. green, blue and purple) one is drawn from the 29 (resp. 604, 348 and 19) samples from which $\hat{g} = 1$ (resp. 2, 3 and 4) groups have been selected.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>Figure 6: Misclassified students rate distribution with respect to the number *n* of students and the selected number $\hat{g}$ of students groups for $\varepsilon = 0.25$. Each color corresponds to a given selected number of students groups ($\hat{g} = 1, 2, 3, 4$). The sample size *n* is followed by the numbers of samples for each boxplot, when these numbers are greater or equal to 10. For example, the first four boxplots correspond to *n* = 20. The red (resp. green, blue and purple) one is drawn from the 237 (resp. 635, 126 and 2) samples from which $\hat{g} = 1$ (resp. 2, 3 and 4) groups have been selected.</figcaption></figure>
# 6. Conclusion

In this paper, we have proposed a model selection procedure for latent block models on binary data in the context of placement tests. We have examined the effect of initial values in the combined V-Bayes Gibbs algorithm on this procedure.

Our first results show that this effect is depending on the difficulty of the considered case. In simple cases, initial values have very small effects, which implies that only one initialization in the estimation algorithm is sufficient. On the contrary, in more difficult cases such as the English SELF test data set, initial values can have very important effects and lead to an inappropriate model selection. In such cases, we advice to strongly increase the number of estimation algorithm initializations. In the English SELF test data set, although the hyperparameter $a = 4$ is chosen in order to avoid empty classes, we have got two small probabilities values $\hat{\rho}_1 = 0.137$ and $\hat{\rho}_4 = 0.117$. With a smaller value of $a$, the number of initializations might have been reduced but with a higher risk of empty classes ([5]).

In a second part, we have studied the robustness of our model selection procedure by investigating the selected number of students groups and the stability of the obtained students partitions with respect to the sample size. As expected, the number of selected students groups tends to the reference one as the sample size increases. Moreover, the rate of misclassified students decreases.

In the English SELF test data set, we can observe in Table 4 one items group for which the probabilities of correct answer are similar whatever the students groups. In such a case, co-clustering methods may fail to provide a meaningful result due to the presence of noisy or irrelevant features. As a perspective, it could be relevant to consider another class of models namely the latent blocks models with noise class proposed by Laclau and Brault [16]. This class of models would enable to include a group of items for which all students would have the same probability of success regardless of the group they belong to. It would make it possible to isolate items which have small impact on the students classification and would speak in favour of the removal of these items in the placement tests.
## Acknowledgment

This work has been carried out in the framework of the IRS project COPOLangues funded by IDEX Université Grenoble Alpes. It has also been partially supported by MIAI@Grenoble Alpes (ANR-19-P3IA-0003). All the computations presented in this paper were performed using the GRICAD infrastructure which is supported by Grenoble research communities. The authors thank Sylvain Coulange and Marie-Pierre Jouannaud for the data acquisition and Margaux Leroy for the preliminary robustness study in latent block models with application to an English SELF placement test.

## References

### References

[1] C. Cervini, M. Masperi, M.-P. Jouannaud, F. Scanu, Defining, modeling and piloting SELF, a new formative assessment test for foreign languages, in: J. Colpaert, M. Simons, A. Aerts, M. Oberhofer (Eds.), Language Testing in Europe: time for a new framework, University of Antwerp, 2013.

[2] V. Brault, S. Coulange, F. Letué, M.-P. Jouannaud, M.-J. Martinez, A.-C. Perret, Comment former des groupes d'étudiants homogènes à partir des résultats de SELF ? Présentation d'un outil d'aide à la décision pour la création de groupes, Mediazioni. Rivista online du studi interdisciplinari su lingue e culture 32 (2021) 185-203.

[3] G. Govaert, M. Nadif, Clustering with block mixture models, Pattern Recognition 36 (2) (2003) 463-473.

[4] G. Govaert, M. Nadif, Block clustering with Bernoulli mixture models: Comparison of different approaches, Computational Statistics and Data Analysis 52 (6) (2008) 3233-3245.

[5] C. Keribin, V. Brault, G. Celeux, G. Govaert, Estimation and selection for the latent block model on categorical data, Statistics and Computing 25 (6) (2015) 1201-1216.

[6] A. P. Dempster, N. M. Laird, D. B. Rubin, Maximum likelihood from incomplete data via the em-algorithm, Journal of the Royal Statistical Society: Series B 39 (1) (1977) 1-22.
[7] H. Akaike, Information theory and an extension of the maximum like-
lihood principle, in: Proceedings, 2nd Internat. Symp. on Information
Theory, 1973, pp. 267–281.

[8] H. Akaike, A new look at the statistical model identification, IEEE transactions on automatic control 19 (6) (1974) 716-723.

[9] G. Schwarz, et al., Estimating the dimension of a model, The annals of statistics 6 (2) (1978) 461–464.

[10] C. Biernacki, G. Celeux, G. Govaert, Assessing a mixture model for clustering with the integrated completed likelihood, Pattern Analysis and Machine Intelligence, IEEE Transactions on 22 (7) (2000) 719–725.

[11] C. Keribin, V. Brault, G. Celeux, G. Govaert, Model selection for the binary latent block model, in: 20th International Conference on Computational Statistics (COMPSTAT 2012), Limassol, Cyprus, 2012, pp. 379–390.

[12] C. Kullback, R. A. Leibler, On information and sufficiency, The Annals of Mathematical Statistics (1951) 79–86.

[13] V. Brault, C. Keribin, M. Mariadassou, Consistency and asymptotic nor-
mality of latent block model estimators, Electronic Journal of Statistics
14 (1) (2020) 1234–1268.

[14] V. Robert, Y. Vasseur, V. Brault, Comparing high-dimensional parti-
tions with the Co-clustering Adjusted Rand Index, Journal of Classifi-
cation 38 (2021) 158–186.

[15] A. Lomet, G. Govaert, Y. Grandvalet, Un protocole de simulation de données pour la classification croisée, in: 44e Journées de Statistique, SFdS, Bruxelles, Belgium, 2012, pp. 1–6.

[16] C. Laclau, V. Brault, Noise-free latent block model for high dimensional data, Data Mining and Knowledge Discovery 2 (2019) 446–473.