# Problems of clustering and recent advances

Edwin Diday

► To cite this version:

Edwin Diday. Problems of clustering and recent advances. [Research Report] IRIA-RR-337, IRIA. 1979, pp.19. hal-04716623

HAL Id: hal-04716623

https://inria.hal.science/hal-04716623v1

Submitted on 1 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
liv. 05500 RR 337

(115)

RIA
laboria

laboratoire de recherche
en informatique
et automatique

PROBLEMS OF CLUSTERING
AND RECENT ADVANCES

Edwin DIDAY

Rapport de Recherche N° 337

19p.

Janvier 1979

Institut de Recherche
d'Informatique
et d'Automatique

Domaine de Voluceau
Rocquencourt
B. P. 105 78150 - Le Chesnay
France
Tél.: 954 90 20
# PROBLEMS OF CLUSTERING AND RECENT ADVANCES

## Edwin Diday

### Résumé :

La classification automatique a été souvent présentée comme un ensemble de techniques dont le principal mérite est de donner des classes d'objets. Pour adopter un point de vue différent, on est amené à se poser les deux questions suivantes :

1) quels sont les principaux choix de base que l'utilisateur doit faire avant toute classification ?

2) est-il possible de montrer que la plupart des problèmes classiques de classification peuvent s'exprimer sous forme d'un critère à optimiser ?

On aborde ensuite le problème des tableaux de grande taille. On montre que dans ce cas, beaucoup de problèmes de la statistique classique (estimation de densité, régression, discrimination, analyse factorielle, analyse canonique, etc ...) peuvent être enrichis en considérant le point de vue local apporté par la classification automatique.

### Abstract :

The clustering problem consists in finding and ordering groups of objects, using the relations defined by the variables which characterize these objects. Traditionally, this field has been presented as a set of techniques which were developed for the sole purpose of finding clusters. We adopt here a different point of view.

We shall briefly go over the three following points : 1) What are the basic choices that the user who wishes to classify multidimensional data will have to make at the beginning ?

1) We then show that most problems arising in applications can be formulated in terms of the optimization of a mathematically definable criterion.

2) Last, we show that in many statistics problems (probabilistic or not) we can obtain an additional understanding if we take the clustering point of view, which emphasizes local properties which turn out to be important in practical problems.
PROBLEMS OF CLUSTERING AND RECENT ADVANCES*

Edwin DIDAY

* Invited paper at the 11th European Congress of Statistics - OSLO - August 1978.
# OUTLINE

1) Introduction

2) The basic choices
2.1. The field
2.2. The structure
2.3. The representation
2.4. The allocation

3) Clustering problems and optimization
3.1. Basic choices problems
3.2. Existence of the clusters
3.3. Size of the data arrays
3.4. Multicriterion clustering

4) Cluster analysis and the local approach in statistics
4.1. Typological factor analysis
4.2. Mixed distribution detection
4.3. Regression
4.4. Discriminant factor analysis
4.5. Canonical analysis

5) Conclusion

Bibliography
1) **Introduction** :

The clustering problem consists in finding and ordering groups of objects, using the relations defined by the variables which characterize these objects.

Traditionally, this field has been presented as a set of techniques which were developed for the sole purpose of finding clusters. We adopt here a different point of view.

We shall briefly go over the three following points : 1) What are the basic choices that the user who wishes to classify multidimensional data will have to make at the beginning ?

2) We then show that most problems arising in applications can be formulated in terms of the optimization of a mathematically definable criterion.

3) Last, we show that in many statistics problems (probabilistic or not) we can obtain an additional understanding if we take the clustering point of view, which emphasizes local properties which turn out to be important in practical problems.

2) **The basic choices** :

2.1. The field of data

After we have defined the set of objects E, we have to choose the variables which characterize them, discrete or continuous, quantitative or qualitative, ordinal or nominal ; next, there is the problem of encoding these variables and defining a measure of similarity between objects.

Various authors have studied the problems related to these choices. The problem of the determination of the most classifying variables has been considered by Aivazian [1976]. Once the sets of objects and variables are well defined, another important problem is how to encode the variables ; for instance how to transform a quantitative variable into a qualitative one, (see Anderberg [1973], Lechevallier [1974]). The problem of adapting the measure of similarity to the form of the clusters to determine, has also been studied by several authors (Rohlf [1970], Chernof [1970], Diday and Govaert [1974]).
## 2.2. The structure

Many kinds of structures. S can be imagined, the most useful in practice are actually :

S = $\mathbb{P}_k$ the set of partitions of E in k classes.

S = H the set of hierarchies on E (i.e. h ∈ H verifies the following properties :

E ⊆ h, E ∈ h, h₁, h₂ ∈ h => {h₁ ∩ h₂ = ∅, h₁ ⊂ h₂ or h₂ ⊂ h₁}.

S = O the set of overlapping clusters of E.

## 2.3. The representation

2.3.1. The representation of a cluster is an expression with respect to which all the objects can be located. (Four examples are given on Figure 1) This can be made in the two following steps :

a) the choice of the set $\mathbb{I}$ of the possible representations (for example $\mathbb{I} = \mathbb{R}^p$, if p is the number of variables characterizing the objects).

b) the choice of a map g : $\mathbb{P} \to \mathbb{I}$ which associates to each cluster of objects a unique representation (for example g(A) = G, the center of gravity of the set of objects of cluster A, considered as points in $\mathbb{R}^p$).

2.3.2. The visual representation of the set of clusters depends on the structure of these clusters. The figure 2 shows two kinds of visual representation : (a) a hierarchy (b) partitions on a factorial plane.

## 2.4. The allocation

Several clusters of objects being constituted, we then have to define a way to allocate an object to one of these clusters. This can be done in two steps : a) the choice of a measure of similarity $D(x,A)$ between an object x and the representation of a cluster A. b) the choice of an allocation function to one of the clusters.

These two steps can be summarized into the map $f : \mathbb{I}_k \to \mathbb{P}_k$ which associates to k representations $L = (L_1, ..., L_k)$ a partition $P = (P_1, ..., P_k)$, that is $f(L) = P$. The map $f$ may be defined in the following manner :

$P_i = \{ x \in E \mid D(x, L_i) < D(x, L_j) \}$ and in case of equality, x is allocated to the cluster of smaller index.
### 3. Clustering problems and optimisation

#### 3.1. Basic choices problems:

With each basic it is possible to associate an optimization problem
Let us give some examples of such problems.

##### 3.1.1. Optimisation in the field of data

In the field of data we have to choose the variables and the measure
of resemblance. So we may set two following problems :

a) Given a set of variables of size N, how to select the best variables for clustering ? We can set for instance (see Diday 1978) the following optimisation problem :

$$ \text{Minimize } W (Y, P) = \sum_{i=1}^{k} \sum_{j=1}^{n} \text{var}_j (P_i) \quad (1) $$
$$ \text{POS} \\ \text{YOL} $$

where $\text{var}_j P_i$ is the variance of $P_i \subset E$ on the variable $y_j$.

S a set of admissible structures. $P = (P_1, ..., P_k)$ where $P_i \subset E, \mathcal{L}$ the subsets of variables, of size $n < N$.

**Example** : Let us give $N = 4, n = 3, S = P_2$ where $P_2$ is the set of the partitions into two classes of E.

<table><thead><tr><th></th><th>y<sub>1</sub></th><th>y<sub>2</sub></th><th>y<sub>3</sub></th><th>y<sub>4</sub></th></tr></thead><tbody><tr><th>x<sub>6</sub></th><td>2</td><td>1</td><td>5</td><td>10</td></tr><tr><th>x<sub>2</sub></th><td>3</td><td>1</td><td>5</td><td>5</td></tr><tr><th>x<sub>3</sub></th><td>2</td><td>1</td><td>6</td><td>1</td></tr><tr><th>x<sub>1</sub></th><td>9</td><td>2</td><td>3</td><td>2</td></tr><tr><th>x<sub>4</sub></th><td>7</td><td>2</td><td>4</td><td>6</td></tr><tr><th>x<sub>5</sub></th><td>8</td><td>2</td><td>2</td><td>9</td></tr></tbody></table>

The solution which optimizes the criterion (1) and which corresponds to the above mentioned array, is : $Y^* = (y_1, y_2, y_3)$ and $P^* = (P_1^*, P_2^*)$ with $P_1^* = \{x_6, x_2, x_3\}, P_2^* = \{x_1, x_4, x_5\}$.
b) Given a set of admissible measures of resemblance D, how to select the best
one for clustering ? We can for instance define the following criterion :

$$
\begin{array}{l c c c c}
 & k & & & \\
\text{Minimize} & \sum_i \sum_{j \in P_i} & d(x,y) & & \\
\text{d} \in D & i=1 & x \in P_i & & \\
\text{P} \in S & & y \in P_i & & \\
\end{array}
$$

(see Diday et al 1978).

3.1.2. Optimization of the structure

Whichever the choice of the structure (hierarchy, minimum spanning
tree, partition, overlapping clusters, etc...), we have to make sure that
the distortion of the initial data is minimized. Several authors have pro-
posed distortion indexes between the measure of similarity defined on the
field and the one induced by the chosen structure, see Cormack [1971]. If,
for example, the chosen structure is a hierarchy, then, the distortion has
to be minimized over all the ultrametrics. See Caroll et al. [1975] for
the hierarchy case, Regnier [1965] and Mirkine [1977] for the partition case.
Another way to improve the chosen structure consists in optimizing the
adequacy between the clusters defined by the structure and the mode of re-
presentation (see Diday [1972]). If, for example, the clusters are repre-
sented by their center of gravity, a simple criterion consists in minimizing
the variance or the inertia of the clusters (see Ward [1968], Beale [1969],
Diday [1976]).

Example 1

If at each level of a hierarchy h we assign a compatible positive value
we obtain an "indexed hierarchy". An indexed hierarchy h induces a measure
of resemblance d_h (which is an ultrametric).

<figure><img src="image_2.png" /></figure>
The optimisation problem is to find the structure which minimizes the
distortion of the initial field of data. More precisely this problem can
be expressed as follows :

$$
\begin{array}{ll}
\text{Minimize} & W (d_h, d) \\
\text{h} \in \mathcal{H}_I &
\end{array}
$$

where d is the measure of ressemblance choosen in the field of data.

$H_I$ the set of all the indexed hierarchies on E.

W an indexe which measure the distortion between d and d<sub>h</sub>. For instance, by
ordering the value taken by d and d<sub>h</sub> we obtain two orders, then W can be taken
as the Kendall's τ of these orders.

Example 2

$$
\begin{array}{ll}
\text{Minimize} & W(s,R) \\
s \in S \\
R \in \mathcal{P}(\mathcal{L})
\end{array}
$$

where $\mathcal{L}$ is a set of admissible representations ; $\mathcal{P}(\mathcal{L})$ is the power set of $\mathcal{L}$ ;
$\mathbb{R}$ is a set of representations associated to the clusters induced by $s$.

For instance, let $S = \mathbb{P}_k$ and $\mathcal{L} = \mathbb{R}^N$ ($N$ is the number of variables which
characterize the objects of $E$) ; it is easy to show that the problem amount to
look for the partition which

$$
\begin{array}{ll}
\underset{P \in \mathbb{P}_k}{\text{Minimize}} & W(P) = \sum_{i=1}^{k} \text{var } P_i.
\end{array}
$$

3.2. Existence of the clusters

It may be very useful to partition a set of objects into clusters,
even if their distribution into the space of variables is uniform. However,
it is legitimate to wonder wether the population of objects is heterogeneous
and wether the clusters obtained have a real existence. A probabilistic approach
to these problems has been recently proposed by Sneath [1977] and Bock [1977].
### 3.3. Size of the data arrays

With the continuing progress of computer technology, greater and greater quantities of information can be stored, and some users may be confronted with clustering problems involving more than 40,000 customers, characterized by 100 variables (for mail order sales, for example). The classical criterion to optimize and associated algorithms, must therefore be adapted, (see Bruynooghe [1977], Diday [1975], for the deterministic approach, and Mac Queen [1967], Hartigan [1977], for the probabilistic approach).

### 3.4. Multicriterion clustering

This class of problems arises frequently in certain fields. For example, a demographic study is involved with the problem of finding the groups of towns which satisfy the three following criteria : the deviation to the mean of the variables characterizing these towns must be as small as possible ; the size of the population must be constant in each group ; the towns in each group must satisfy a geographical connectedness criterion.

See Monestiez [1977], Lebart et Roche [1978] . In life sciences, or in botanical sciences, species must be classified according to their similarities as well as their evolution. "Find the most parsemonious (minimum length) phylogenetic tree needed to establish the phenetic pattern of the taxa".

See Sneath and Sokhal [1973].

An interesting multicriterion problem in clustering is the following :
find simultaneously clusters of objects and clusters of variables (see
Hartigan [1976], Govaert [1977], Diday [1976]), which optimize both a criterion
depending on the objects and a criterion depending on the variables.
### 4) Cluster analysis and the local approach in statistics

When the size of the population to classify gets large, a given statistical model may be wrong for the whole population, but very accurate, for certain of its parts. To approach this problem, we have used the so-called notion of "nuées dynamiques", which consists in defining first a criterion $W(L,S)$, which measures the adequacy between the representation $L$, called "Kernel", and the chosen classification structures, (generally, a partition). Then, an algorithm is defined, based on the choice of two functions $g$ and $f$, characterizing the representation and allocation processes. If $g$ and $f$ are properly chosen, it can be proved that the criterion $W$ improves at each iteration and converges to a stable position. The main interest of this method is to unify a large variety of problems, as well as to give algorithms leading to a (locally optimal) solution, with the help of a computer. Let us give now five examples which illustrate this approach.

#### 4.1. Typological factor analysis

In Diday [1974] a generalization of the classical factor analysis to the simultaneous determination of $k$ varieties of minimum inertia, instead of one, is given. The structure $S$ is a partition $P = (P_1, ..., P_k)$ and $L = (L_1, ..., L_k)$, where $L_i$ is a variety which define the representation of $P_i$. The constructed algorithm converges towards a solution which tends to minimize the local inertia with respect to the local variety $L_i$ defined by the subset $P_i$. The local factorial planes (or axes) obtained, explain a more important part of the inertia than the classical global factorial planes (and axes), see Figure 3.
## 4.2. Decomposition of mixed distribution

This method is applied to detect, in a given multivariate sample, the possible presence of sub-samples, drawn from probability distributions of some known type. The chosen structure is a partition ($P_1, ..., P_k$) and the kernels $L_i$ are distribution functions of the same family. Gaussian in Wolf [1970] and Diday and al [1974] or Erlang in Schroeder [1976]. The criterion measures the likelihood of $B_i$ knowing $L_i$ (see Fig. 4).

## 4.3. Regression

We consider here, the problem of discontinuous shifts in regression regimes, at unknown points in the data series.

The structure is a partition $P = (P_1, ..., P_k)$ and $L = (L_1, ..., L_k)$ where $L_i$ are the coefficients of the local regression, associated to $P_i$. The criterion measures, for example, the sum of the mean square error of k local regressions (see Diday [1976] and Charles [1977]). For the case when the structures is a hierarchy, see Mc Gee [1970].

## 4.4. Discriminant factor analysis

The problem consists in finding the best discriminant factors of the clusters to determine. The structure S is a partition and the unique kernel L is a discriminant factor plane (see figure 5 and Diday [1976]), Diday et al (1978)).

## 4.5. Canonical analysis

When data arrays have a large size, it is legitimate to look for linear combinations depending on the local tendencies which may appear in the population. Both these tendencies and the canonical components which are associated to them in the "best" possible way must be simultaneously detected. We present in Diday [1978], several algorithms in order to minimize the criterion, in both the cases when the data are centered or not. In the case where all the variables are qualitative, the problem is posed in terms of the factor analysis of correspondances and it amounts to look for the clusters of objects, that induce the largest $\chi^2$ on the contingency between the variables. If one of the two variables blocks is composed of incidence variables, we arrive to interesting methods of local discriminant analysis, see figure 6.
Mathematical formalisation of the problem :

The data array is defined as follows :

<figure><img src="image_3.png" /></figure>

$$
\text{Let } \xi_i = X_i^T L_i, \eta_i = Y_i^T b_i.
$$

The problem can be set as follows :

Find $L_i \in \mathbb{R}^p$ and $b_i \in \mathbb{R}^q$ which maximize :

$$
\sum_{i=1}^{k} \operatorname{cor}^2 (\xi_i, \eta_i) = \sum_{i=1}^{k} \frac{\langle \xi_i, \eta_i \rangle^2}{\|\xi\|^2_{M_i} \|\eta_i\|^2_{M_i}}
$$

To minimize $\Sigma_{i=1}^k \operatorname{cor}^2(\xi_i, \eta_i)$ is equivalent to minimize :

The following criterion :

$$
W (L,P) = \sum_{i=1}^{k} \frac{\| X_i' L_i - Y_i' V_{i22}^{-1} V_{i21} L_i \|^2}{\| a_i \|^2_{V_{i11}}}
$$

$$
\text{where } V_{i11} = X_i^T M_i X_i', \quad V_{i22} = Y_i^T M_i Y_i', \quad V_{i21} = Y_i^T M_i X_i',
$$

$$
M_i = \frac{1}{n_i} \quad I_i, \quad I_j = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \otimes n_i, \quad \langle n_i, \cdot \cdot \cdot n_i \rangle
$$
## 5. Conclusion

Clustering is often considered as a set of algorithms whose main merit is to give clusters ! In fact, after a more detailed analysis, we can see that there are very few clustering algorithms that deal with the same problems and that optimize the same criterion. For these reasons, we have willingly neglected here, the algorithmic aspect in order to point out a certain number of problems frequently encountered in clustering, as well as the underlying criteria to optimize. In order to clearly distinguish both aspects (Problem and algorithm), we have shown how in classical statistics, clustering problems could appear without leading to algorithms to solve them.
BIBLIOGRAPHY

AIVAZIAN S. [1976] "Les méthodes statistiques d'étude de dépendances entre variables classifiantes", T.S.E.M.I. Moscou URSS.

ANDERBERG, M.R. [1973] "Cluster Analysis for Applications" Academic Press.

BENZECRI, J.P. [1973] "L'analyse des données", Tome 1 et Tome 2, Paris, Dunod.

BEALE, E.M.L. [1969] "Euclidean cluster analysis Bull I.S.I., 43 Book 2. 92-94 (London).

BOCK, H.H. [1977] "On tests concerning the existence of a classification", Proc. 1 th Int. Symp. on Data Analysis and Informatics, IRIA, Rocquencourt, 78150 FRANCE.

BRUYNOOGHE, M. [1977] "Classification automatique d'un grand ensemble de données par la méthode des graphes réductibles", Proc. 1 th. Int. Symp. on Data Analysis and Informatics, IRIA, Rocquencourt, 78150 FRANCE.

CHARLES, C. [1977] "Représention typologique", Rapport de recherche n° 257. IRIA-LABORIA, Rocquencourt, 78150 FRANCE.

CAROLL, J.D., S. PRUZANSKY, S. [1975] "Fitting of hierarchical tree structure", US-Japan Sem. Univ. of California, San Diego, USA.

CHERNOFF, M. [1970] "Metric consideration in cluster analysis" Proc. 6 th. Berkeley Symposium on Math. Stat. and Prob.

CORMACK, R.M. [1971] "A review of classification", J. Royal Statist. Soc., Serie A, 134, 321.

DIDAY, E. [1974] "L'analyse factorielle typologique" R.S.A. vol. XXII n° 4.

DIDAY, E. [1975] "Classification automatique séquentielle pour grands tableaux" RAIRO, mars, B1.
DIDAY, E. [1978] "Canonical analysis and clustering", Université Paris IX-Dauphine, Cahiers du CEREMADE.

DIDAY, E. [1978] "Selection of variables and clustering", The Fourth International Joint Conference on Pattern Recognition, Kyoto, Japan.

DIDAY, E. SCHROEDER, A. OK, Y. [1974] "The dynamic clusters method in Pattern Recognition", In : Information Processing 74 ; IFIP, 691.

DIDAY, E. GOVAERT, G. [1974] "Adaptative resemblance measure in Pattern Recognition" ; Computer oriented learning process - NATO, ASI Bonas (1974).

DIDAY, E. GOVAERT, G. LEMOINE, Y. [1978] "A new kind of representation in clustering", The Fourth International Joint Conference on Pattern Recognition, Kyoto, Japan.

GOVAERT, G. [1977] "Algorithme de classification d'un tableau de contingence" in 1st Symp. on data Analysis, IRIA Rocquencourt, 78150 FRANCE.

HARTIGAN, J.A. [1977] "Clusters as Modes", in : 1st int. symp. on Data Analysis and informatics, IRIA Rocquencourt, 78150 FRANCE.

LERMAN, I.C. [1970] "Les bases de la classification automatique", Paris, Gauthiers-Villars.

LEBART, L. et ROCHE C. [1978] "Classification visualisée de grands ensembles sous double contrainte", 2° Journées Nationales de Classification, Nice.

LECHEVALLIER, Y. [1976] "Classification automatique sous contrainte d'ordre total" Rapport de Recherche IRIA-LABORIA n° 200.

MIRKIN, B.G. [1977] "Problems of formalization in the social sciences", Institut d'Economie Akademgorod, USSR
Mc. QUEEN, J. [1967] "Some Methods for classification and analysis of Multivariate observations", Proc. 5 th. Berkeley Symposium 1, 281-297.

Mc GEE and CARLTON [1970] "Piecewise Regression" JASA, 65, Sept.

MONESTIEZ, P. [1977] "Methodes de classification automatique sous contraintes spatiales" 1ère Journées Nationales de classification (Vannes).

REGNIER, S. [1965] "Sur quelques aspects mathématiques des problèmes de classification" ICC Bull. Vol. 4, 175-191.

ROHLF, F.J. [1970], "Adaptative Hiearachical Clustering Schemas", Systematic Zoology, 18, 58-82.

SNEATH, P.H.A. [1977] "Cluster Significance tests and their relation to measures of overlap", in : 1st int. Symp. on Data Analysis and informatics, IRIA Rocquencourt, 78150 FRANCE.

SCHROEDER, A. [1976] "Analyse d'un mélange de distributions", R.S.A. Vol XXIV, n° 1.

SNEATH, P.H.A., SOKAL, R. [1973] Numerical Taxonomy. Freeman, San Francisco.

WARD, J.H. [1963] "Hierarchical Grouping to optimize and objective function" JASA, 58, 236-244.

WOLF, J.H. [1970] "Pattern clustering by Multivariate Mixture analysis", Multivariate Behavioral Research 5, 329-350.
<figure><img src="image_4.png" /><figcaption>(a)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b)</figcaption></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /><figcaption>(c)</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(d)</figcaption></figure>

$$L = (\mu, \Sigma)$$

<caption>Figure 1</caption>

(a) : The cluster is represented by its center of gravity

(b) : The cluster is represented by the two closest points to its center of gravity.

(c) : The cluster is represented by the line of least-inertia.

(d) : The cluster A is represented by a normal distribution ; its parameters $L = (\mu, \Sigma)$ are estimated by the sub-sample defined by A.

<figure><img src="image_9.png" /><figcaption>(a)</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>(b)</figcaption></figure>

<caption>Figure 2</caption>

(a) hierarchical representation of clusters

(b) partitions on a factorial analysis plane
<figure><img src="image_11.png" /><figcaption>Figure 3
Typological factor analysis</figcaption></figure>

Find the three local factor planes $(L_1), (L_2), (L_3)$, such that they explain a more important part of inertia, then the global one (d).

<figure><img src="image_12.png" /><figcaption>Figure 4
Mixed distribution detection</figcaption></figure>

Under the assumption that the population is a sample of three gaussian distribution (see (a)), find simultaneously the tree clusters $(P_1, P_2, P_3)$ and the three gaussian distribution $(L_1, L_2, L_3)$ which maximize the likelihood of $P_i$ knowing $L_i$ (see (b)).
<figure><img src="image_13.png" /><figcaption>Figure 5
Find the best discriminant plane of the clusters that have to be determined</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure 6</figcaption></figure>

The clustering point of view in canonical analysis can be applied to discriminant analysis as follows : find the partition ($P_1, P_2$) and linear combinations ($L_1, L_2$) of the variables having the best separating power of the a priori given clusters $C_1, C_2, C_3, C_4$.