Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Multivariate Data Analysis Special focus on Clustering and Multiway Methods

François Husson & Julie Josse Applied mathematics department, Agrocampus Rennes 

### useR! 2010, July 20, 2010

1 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Why a tutorial on Multivariate Data Analysis?

• 

### Our research focus is principal component methods
• 

### We teach multivariate data analysis
• 

### We have developed R packages:
• FactoMineR to perform principal component methods • PCA, correspondence analysis (CA), multiple correspondence analysis (MCA), multiple factor analysis (MFA) • complementarity between clustering and principal component methods • missMDA to handle missing values in and with multivariate data analysis • perform principal component methods (PCA, MCA) with missing values • simple and multiple imputation based on principal component models for continuous and categorical data 2 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Outline Multivariate data analysis with a special focus on clustering and

multiway methods
1 

### Principal Component Analysis (PCA) 2 

### Multiple Factor Analysis (MFA) 3 

### Complementarity between Clustering and Principal Component methods ⇒ Multidimensional descriptive methods ⇒ Graphical representations
3 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

# Principal Component Analysis

1 

### Data - Issues - Preprocessing 2 

### Individuals Study 3 

### Variables Study 4 

### Helps to Interpret

4 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Principal Component Analysis Dimensionality reduction ⇒ describes the dataset with a smaller number of variables Technique widely used for applications such as: data compression, data reconstruction, preprocessing before clustering, and ... Descriptive methods

5 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## PCA deals with which kind of data? PCA deals with continuous variables, but categorical variables can also be included in the analysis

Figure: Data table in PCA 

### Many examples: • 

### Sensory analysis: products - descriptors • 

### Ecology: plants - measurements; waters - physico-chemical analyses • 

### Economy: countries - economic indicators • 

### Microbiology: cheeses - microbiological analyses • 

### etc.

6 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Wine data
• 

### 10 individuals (rows): white wines from Val de Loire • 

### 30 variables (columns): • 27 continuous variables: sensory descriptors • 2 continuous variables: odour and overall preferences • 1 categorical variable: label of the wines (Vouvray - Sauvignon)

7 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Problems - objectives

• 

### Individuals study: similarity between individuals with respect to all the variables ⇒ partition between individuals • 

### Variables study: linear relationships between variables ⇒ visualization of the correlation matrix (denoted S ); nd synthetic variables • 

### Link between the two studies: characterization of the groups of individuals by the variables; specic individuals to better understand links between variables

8 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Two clouds of points

Figure: Two clouds of points 9 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Preprocessing ⇒ Similarity between individuals: Euclidean distance • 

### Choosing active variables

d2 

### ( i , i0 

### ) = K



### X k = 1 

### (x ik 

### − x i0 k 

### ) 2

• 

### Variables are always centred

d2 

### ( i , i0 

### ) = K



### X k = 1 

### ((x ik 

### −  ̄ x k 

### )

− (x

i0 k 

### −  ̄ x k 

### )) 2

• 

### Standardizing variables or not?

d2 

### ( i , i0 

### ) = K



### X k = 1 

### 1 s2 k 

### (x ik 

### − x i0 k 

### ) 2
10 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Individuals cloud

• 

### Study the structure, i.e. the shape of the cloud of individuals • 

### Individuals are in R K
11 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Fit the individuals cloud Find the subspace which better sums up the data

Figure: Camel vs dromedary? 

### ⇒ Closest representation by projection ⇒ Best representation of the diversity, variability 12 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Fit the individuals cloud

Pu1 

### (x i . 

### ) = u1 

### ( u0

1 

### u1 

### ) − 1 

### u0

1 

### x i . 

### = < x

i . 

### , u1 

### > u1 

### F i 1 

### = < x

i . 

### , u1 

### > • 

### Minimize the distance between individuals and their projections • 

### Maximize the variance of the projected data u1 

### = arg max u1 ∈ R K 

### ( var ( F

. 1 

### )) = arg max

u1 ∈ R K 

### ( var (Xu1 

### )) with u0 1 

### u1 

### = 1 ⇒ u1 

### rst eigenvector of the correlation matrix associated with the largest eigenvalue λ 1 

### : Su1 

### = λ 1 

### u1 

### Var ( F

. 1 

### ) = var (Xu1 

### ) = 1 / I u0 1 

### X0

### Xu1 

### = u0

1 

### Su1 

### = λ 1 

### u0

1 

### u1 

### = λ 1 13 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Fit the individuals cloud Additional axes are sequentially dened: each new direction maximizes the projected variance among all orthogonal directions ⇒ Q eigenvectors u1 

### ,..., u Q 

### associated to λ 1 

### ,..., λ Q 

### Representation quality: dimensionality reduction ⇒ loosing information
• 

### Total variance of the initial individuals cloud (total inertia):

1 I k x i . 

### − g k 2 

### = tr ( S ) = K



### X k = 1 

### λ k 

### (= K)

• 

### Variance of the projected individuals cloud (Q-dimensional representation): var ( F 1 

### ) + var ( F

2 

### ) + ... + var ( F

Q 

### ) ⇒ Percentage of variance explained: P Q k = 1 λ k P K k = 1 λ k 14 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Example: wine data
• 

### Sensory descriptors are used as active variables: only these variables are used to construct the axes • 

### Variables are (centred and) standardized

15 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Example: graph of the individuals

⇒ Need variables to interpret the dimensions of variability 16 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Individuals coordinates considered as variables

17 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Interpretation of the individuals graph with the variables
• 

### Correlation between variable x
. k 

### and F

. 1 

### (and F

. 2 

### )

⇒ Correlation circle 18 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Interpretation of the individuals graph with the variables

19 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Cloud of variables

Since variables are centred: cos ( θ kl 

### ) = < x

. k 

### , x
. l 

### > k x
. k 

### k k x
. l 

### k = P I

i= 1 

### x ik

### x il 

### q( P I i= 1 

### x2 ik 

### )( P I i= 1 

### x2 il 

### ) = r (x

. k 

### , x
. l 

### ) 20 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Fit the variables cloud Find v 1 

### (in R I 

### , with v0

1 

### v 1 

### = 1) which best ts the cloud

Pv 1 

### (x
. k 

### ) = v

1 

### ( v0 1 

### v 1 

### ) − 1 

### v0 1 

### x
. k 

### G k 1 

### = 1 / I < v

1 

### , x . k > G k 1 

### = 1 / I < v

1 

### , x . k > k v 1 

### k k x
. k 

### k arg max v 1 ∈ R I K 

### X i= k 

### G 2 k 1 

### = arg max v 1 ∈ R I K 

### X i= k 

### r ( v 1 

### , x
. k 

### ) 2


### ⇒ v 1 

### is the best synthetic variable ⇒ v 1 

### , ..., vQ 

### are the eigenvectors of W = XX0 

### the inner product matrix associated with the largest eigenvalues: Wv q 

### = λ q 

### v

q 21 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Fit the variables cloud

⇒ Same representation! What a wonderful result! 22 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Projections... r (A , B ) = cos ( θA , B 

### ) cos ( θA , B 

### ) ≈ cos ( θ H A , HB 

### ) if variables are well projected

Only well projected variables can be interpreted! 23 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Link between the two representations: transition formulae

24 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Link between the two representations: transition formulae

• 

### Su = X0

### Xu = λ u • 

### XX0

### Xu = X λ u → W (Xu ) = λ (Xu ) • 

### WF = λ F and since Wv = λ v then F and v are collinear • 

### Since, || F || = λ and || v || = 1 we have:

v = 1 √λ 

### F ⇒ G = X0 

### v = 1 √λ 

### X0 

### F u = 1 √λ 

### G ⇒ F = Xu = 1 √λ 

### XG F iq 

### = 1 pλ q K 

### X k = 1 

### x ik 

### G kq 

### G

kq 

### = 1 pλ q I 

### X i= 1 

### x ik 

### F iq 

### F

. q 

### : principal components, scores G

. q 

### : correlations between variables and principal components 25 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Link between the two representations: transition formulae

F iq 

### = 1 pλ q K 

### X k = 1 

### x ik 

### G kq 

### G

kq 

### = 1 pλ q I 

### X i= 1 

### x ik 

### F iq 

### What does it mean? An individual is at the same side as the variables for which it takes high values

Figure: Individuals and variables representations 26 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Supplementary information
• 

### For the continuous variables: projection of supplementary variables on the dimensions • 

### For the individuals: projection • 

### For the categories: projection at the barycentre of the individuals who take the categories

⇒ Supplementary information do not create the dimensions 27 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Choosing the number of componentsBar plot, test on eigenvalues, condence interval, cross-validation (functions estim_ncpPCA and estim_ncp), etc.
Two objectives: ⇒ Interpretation ⇒ Separate structure and noise

28 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Percentage of variance obtained under independence ⇒ Is there a structure on my data? Number of variables nbind 4 5 6 7 8 9 10 11 12 13 14 15 16 5 96.5 93.1 90.2 87.6 85.5 83.4 81.9 80.7 79.4 78.1 77.4 76.6 75.5 6 93.3 88.6 84.8 81.5 79.1 76.9 75.1 73.2 72.2 70.8 69.8 68.7 68.0 7 90.5 84.9 80.9 77.4 74.4 72.0 70.1 68.3 67.0 65.3 64.3 63.2 62.2 8 88.1 82.3 77.2 73.8 70.7 68.2 66.1 64.0 62.8 61.2 60.0 59.0 58.0 9 86.1 79.5 74.8 70.7 67.4 65.1 62.9 61.1 59.4 57.9 56.5 55.4 54.3 10 84.5 77.5 72.3 68.2 65.0 62.4 60.1 58.3 56.5 55.1 53.7 52.5 51.5 11 82.8 75.7 70.3 66.3 62.9 60.1 58.0 56.0 54.4 52.7 51.3 50.1 49.2 12 81.5 74.0 68.6 64.4 61.2 58.3 55.8 54.0 52.4 50.9 49.3 48.2 47.2 13 80.0 72.5 67.2 62.9 59.4 56.7 54.4 52.2 50.5 48.9 47.7 46.6 45.4 14 79.0 71.5 65.7 61.5 58.1 55.1 52.8 50.8 49.0 47.5 46.2 45.0 44.0 15 78.1 70.3 64.6 60.3 57.0 53.9 51.5 49.4 47.8 46.1 44.9 43.6 42.5 16 77.3 69.4 63.5 59.2 55.6 52.9 50.3 48.3 46.6 45.2 43.6 42.4 41.4 17 76.5 68.4 62.6 58.2 54.7 51.8 49.3 47.1 45.5 44.0 42.6 41.4 40.3 18 75.5 67.6 61.8 57.1 53.7 50.8 48.4 46.3 44.6 43.0 41.6 40.4 39.3 19 75.1 67.0 60.9 56.5 52.8 49.9 47.4 45.5 43.7 42.1 40.7 39.6 38.4 20 74.1 66.1 60.1 55.6 52.1 49.1 46.6 44.7 42.9 41.3 39.8 38.7 37.5 25 72.0 63.3 57.1 52.5 48.9 46.0 43.4 41.4 39.6 38.1 36.7 35.5 34.5 30 69.8 61.1 55.1 50.3 46.7 43.6 41.1 39.1 37.3 35.7 34.4 33.2 32.1 35 68.5 59.6 53.3 48.6 44.9 41.9 39.5 37.4 35.6 34.0 32.7 31.6 30.4 40 67.5 58.3 52.0 47.3 43.4 40.5 38.0 36.0 34.1 32.7 31.3 30.1 29.1 45 66.4 57.1 50.8 46.1 42.4 39.3 36.9 34.8 33.1 31.5 30.2 29.0 27.9 50 65.6 56.3 49.9 45.2 41.4 38.4 35.9 33.9 32.1 30.5 29.2 28.1 27.0 100 60.9 51.4 44.9 40.0 36.3 33.3 31.0 28.9 27.2 25.8 24.5 23.3 22.3 Table: 95 % quantile inertia on the two rst dimensions of 10000 PCA on data with independent variables 29 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Percentage of variance obtained under independence
Number of variables nbind 17 18 19 20 25 30 35 40 50 75 100 150 200 5 74.9 74.2 73.5 72.8 70.7 68.8 67.4 66.4 64.7 62.0 60.5 58.5 57.4 6 67.0 66.3 65.6 64.9 62.3 60.4 58.9 57.6 55.8 52.9 51.0 49.0 47.8 7 61.3 60.7 59.7 59.1 56.4 54.3 52.6 51.4 49.5 46.4 44.6 42.4 41.2 8 57.0 56.2 55.4 54.5 51.8 49.7 47.8 46.7 44.6 41.6 39.8 37.6 36.4 9 53.6 52.5 51.8 51.2 48.1 45.9 44.4 42.9 41.0 38.0 36.1 34.0 32.7 10 50.6 49.8 49.0 48.3 45.2 42.9 41.4 40.1 38.0 35.0 33.2 31.0 29.8 11 48.1 47.2 46.5 45.8 42.8 40.6 39.0 37.7 35.6 32.6 30.8 28.7 27.5 12 46.2 45.2 44.4 43.8 40.7 38.5 36.9 35.5 33.5 30.5 28.8 26.7 25.5 13 44.4 43.4 42.8 41.9 39.0 36.8 35.1 33.9 31.8 28.8 27.1 25.0 23.9 14 42.9 42.0 41.3 40.4 37.4 35.2 33.6 32.3 30.4 27.4 25.7 23.6 22.4 15 41.6 40.7 39.8 39.1 36.2 34.0 32.4 31.1 29.0 26.0 24.3 22.4 21.2 16 40.4 39.5 38.7 37.9 35.0 32.8 31.1 29.8 27.9 24.9 23.2 21.2 20.1 17 39.4 38.5 37.6 36.9 33.8 31.7 30.1 28.8 26.8 23.9 22.2 20.3 19.2 18 38.3 37.4 36.7 35.8 32.9 30.7 29.1 27.8 25.9 22.9 21.3 19.4 18.3 19 37.4 36.5 35.8 34.9 32.0 29.9 28.3 27.0 25.1 22.2 20.5 18.6 17.5 20 36.7 35.8 34.9 34.2 31.3 29.1 27.5 26.2 24.3 21.4 19.8 18.0 16.9 25 33.5 32.5 31.8 31.1 28.1 26.0 24.5 23.3 21.4 18.6 17.0 15.2 14.2 30 31.2 30.3 29.5 28.8 26.0 23.9 22.3 21.1 19.3 16.6 15.1 13.4 12.5 35 29.5 28.6 27.9 27.1 24.3 22.2 20.7 19.6 17.8 15.2 13.7 12.1 11.1 40 28.1 27.3 26.5 25.8 23.0 21.0 19.5 18.4 16.6 14.1 12.7 11.1 10.2 45 27.0 26.1 25.4 24.7 21.9 20.0 18.5 17.4 15.7 13.2 11.8 10.3 9.4 50 26.1 25.3 24.6 23.8 21.1 19.1 17.7 16.6 14.9 12.5 11.1 9.6 8.7 100 21.5 20.7 19.9 19.3 16.7 14.9 13.6 12.5 11.0 8.9 7.7 6.4 5.7 Table: 95 % quantile inertia on the two rst dimensions of 10000 PCA on data with independent variables 30 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Quality of the representation: cos2

• 

### For the variables: only well projected variables (high cos2 

### between the variable and its projection) can be interpreted! round(res.pca$var$cos2,2) Dim.1 Dim.2 Odor.Intensity.before.shaking 0.01 0.94 Odor.Intensity.after.shaking 0.01 0.89 Expression 0.11 0.71

• 

### For the individuals: (same idea) distance between individuals can only be interpreted for well projected individuals round(res.pca$ind$cos2,2)

Dim.1 Dim.2 S Michaud 0.62 0.07 S Renaudie 0.73 0.15 S Trotignon 0.78 0.07
31 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Contribution ⇒ Contribution to the construction of the dimension (percentage of variability):

• 

### for each individual: Ctr q 

### ( i) = F2

iq P I i= 1 F2 iq 

### = F2

iq λ q 

### ⇒ Individuals with a large coordinate contribute the most round(res.pca$ind$contrib,2)

Dim.1 Dim.2 S Michaud 15.49 3.10 S Renaudie 15.56 5.56 S Trotignon 15.46 2.43
• 

### for each variable: Ctr q 

### ( k) = G 2

kq λ q 

### = r (x. k , v

q ) 2 λ q 

### ⇒ Variables highly correlated with the principal component contribute the most 32 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Description of the dimensions By the continuous variables: • 

### correlation between each variable and the principal component of rank q is calculated • 

### correlation coecients are sorted and signicant ones are given > dimdesc(res.pca)

$Dim.1$quanti $Dim.2$quanti corr p.value corr p.value O.candied.fruit 0.93 9.5e-05 Odor.Intensity.before.shaking 0.97 3.1e-06 Grade 0.93 1.2e-04 Odor.Intensity.after.shaking 0.95 3.6e-05 Surface.feeling 0.89 5.5e-04 Attack.intensity 0.85 1.7e-03 Typicity 0.86 1.4e-03 Expression 0.84 2.2e-03 O.mushroom 0.84 2.3e-03 Aroma.persistency 0.75 1.3e-02 Visual.intensity 0.83 3.1e-03 Bitterness 0.71 2.3e-02 ... ... ... Aroma.intensity 0.66 4.0e-02 O.plante -0.87 1.0e-03 O.flower -0.89 4.9e-04 O.passion -0.90 4.5e-04 Freshness -0.91 2.9e-04 Sweetness -0.78 8.0e-03 33 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Description of the dimensions By the categorical variables: • 

### Perform a one-way analysis of variance with the coordinates of the individuals ( F

. q 

### ) explained by the categorical variable • a F-test by variable • for each category, a Student's t-test to compare the average of the category with the general mean > dimdesc(res.pca) Dim.1$quali R2 p.value Label 0.874 7.30e-05 Dim.1$category

Estimate p.value Vouvray 3.203 7.30e-05 Sauvignon -3.203 7.30e-05

34 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Practice with R 1 

### Choose active variables 2 

### Scale or not the variables 3 

### Perform PCA 4 

### Choose the number of dimensions to interpret 5 

### Simultaneously interpret the individuals and variables graphs 6 

### Use indicators to enrich the interpretation library(FactoMineR) Expert <- read.table("http://factominer.free.fr/useR2010/Expert_wine.csv", header=TRUE, sep=";",row.names=1) res.pca <- PCA(Expert,scale=T,quanti.sup=29:30,quali.sup=1) res.pca x11() barplot(res.pca$eig[,1],main="Eigenvalues",names.arg=1:nrow(res.pca$eig)) plot.PCA(res.pca,habillage=1) res.pca$ind$coord res.pca$ind$cos2 res.pca$ind$contrib plot.PCA(res.pca,axes=c(3,4),habillage=1) dimdesc(res.pca) write.infile(res.pca,file="my_FactoMineR_results.csv") #to export a list 35 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Practice with GUI source("http://factominer.free.fr/install-facto.r")

36 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Handling missing values: missMDA package

⇒ Obtain the principal components from observed data with an EM-type algorithm • 

### Impute missing values with PCA using imputePCA function (tuning parameter: number of components) • 

### Perform the usual PCA on the completed data set library(missMDA) data(orange) nb.dim <- estim_ncpPCA(orange,ncp.max=5) res.comp <- imputePCA(orange,ncp=2) res.pca <- PCA(res.comp$completeObs)

37 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## MCA: problems - objectives • 

### Individuals study: similarity between individuals (for all the variables) → partition between individuals Individuals are dierent if they don't take the same levels
• 

### Variables study: nd some synthetic variables (continuous variables that sum up categorical variables); link between variables ⇒ levels study • 

### Categories study: • two levels of dierent variables are similar if individuals that take these levels are the same (ex: 65 years and retired) • two levels are similar if individuals taking these levels behave the same way, they take the same levels for the other variables (ex: 60 years and 65 years)
• 

### Link between these studies: characterization of the groups of individuals by the levels (ex: executive dynamic women) 38 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## MCA: a PCA on an indicator matrix • Binary coding of the factors: a factor with K j 

### levels → K

j 

### columns containing binary values, also called dummy variables

d2 

### ( i , i0 

### ) = I

J J 

### X j= 1 K

j 

### X k = 1 

### 1 I k 

### (x ik 

### − x i0 k 

### ) 2 39 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## MCA: the superimposed representation F iq 

### = 1 pλ q 

### X k = 1 

### K x

ik 

### J G kq 

### ⇒ Individual i at the barycenter of its levels G kq 

### = 1 pλ q I 

### X i= 1 

### x ik 

### I k 

### F iq 

### ⇒ Level k at the barycenter of the individuals who take this level

40 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



# Multiple Factor Analysis

1 

### Data - Issues 2 

### Common Structure 3 

### Groups Study 4 

### Partial Analyses 5 

### Example "Doing a data analysis, in good mathematics, is simply searching eigenvectors, all the science of it (the art) is just to nd the right matrix to diagonalize" Benzécri 41 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Multiway data set

Examples with continuous and/or categorical sets of variables: • 

### genomic: DNA, protein • 

### sensory analysis: sensorial, physico-chemical • 

### survey: student health (addicted consumptions, psychological conditions, sleep, identication, etc.) • 

### economy: economic indicators for countries by year 42 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Example: gliomas brain tumors

• 

### Transcriptional modication (RNA), microarrays: 489 variables • 

### Damage to DNA (CGH array): 113 variables

43 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Objectives

• 

### Study the similarities between individuals with respect to all the variables • 

### Study the linear relationships between variables
⇒ taking into account the structure on the data (balancing the inuence of each group)

• 

### Find the common structure with respect to all the groups - highlight the specicities of each group • 

### Compare the typologies obtained from each group of variables (separate analyses)
44 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Balancing the groups of variables MFA is a weighted PCA: • 

### compute the rst eigenvalue λ j 1 

### of each group of variables • 

### perform a global PCA on the weighted data table:

  X 1 

### qλ 1 1 

### ; X 2 

### qλ 2 1 

### ; ... ; X

J 

### qλ J 1 

###   ⇒ Same idea as in PCA when variables are standardized: variables are weighted to compute distances between individuals i and i0

45 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Balancing the groups of variables This weighting allows that: • 

### same weight for all the variables of one group: the structure of the group is preserved • 

### for each group the variance of the main dimension of variability (rst eigenvalue) is equal to 1 • 

### no group can generate by itself the rst global dimension • 

### a multidimensional group will contribute to the construction of more dimensions than a one-dimensional group

46 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Individuals and variables representations

Same representations and same interpretation as in PCA
47 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Groups study

⇒ Synthetic comparison of the groups
⇒ Are the relative positions of individuals globally similar from one group to another? Are the partial clouds similar?
⇒ Do the groups bring the same information?

48 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Similarity between two groups Measure of similarity between groups K j 

### and K

m 

### : Lg 

### ( K j 

### , K m 

### ) = X k ∈ K j 

### X l ∈ K m 

### cov 2 

###   x
. k 

### qλ k 1 

### , x
. l 

### qλ l 1 

###   MFA = weighted PCA ⇒ rst principal component of MFA maximizes J



### X j= 1 

### Lg 

### ( v

1 

### , K j 

### ) = J 

### X j= 1 

### X k ∈ K j 

### cov2 

###   x
. k 

### qλ j 1 

### , v 1 

###   Inertia of K

j 

### projected on v

1 49 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the groups Group j has the coordinates (Lg 

### ( v 1 

### , K j 

### ) , Lg 

### ( v

2 

### , K j 

### ))

• 

### 2 groups are all the more close that they induce the same structure • 

### The 1st dimension is common to all the groups • 

### 2nd dimension mainly due to CGH 0 ≤ Lg 

### ( v

1 

### , K j 

### ) = 1 λ j 1 

### X k ∈ K j 

### cov2 

### (x

. k 

### , v 1 

### )
| {z }

≤ λ j 1 

### ≤ 1

50 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Numeric indicators > res.mfa$group$Lg CGH expr WHO MFA CGH 2.51 0.60 0.46 1.96 expr 0.60 1.10 0.36 1.07 WHO 0.46 0.36 0.50 0.51 MFA 1.96 1.07 0.51 1.91 > res.mfa$group$RV CGH expr WHO MFA CGH 1.00 0.36 0.41 0.90 expr 0.36 1.00 0.48 0.74 WHO 0.41 0.48 1.00 0.53 MFA 0.90 0.74 0.53 1.00 

### Lg 

### ( K j 

### , K j 

### ) = P K

j k = 1 

### ( λ j k 

### ) 2 

### ( λ j 1 

### ) 2 

### = 1 + P K j k = 2 

### ( λ j k 

### ) 2 

### ( λ j 1 

### ) 2 

### • CGH gives richer description (Lg 

### greater) • RV: a standardized Lg 

### • CGH and expr are not linked (RV=0.36) • CGH closest to the overall (RV=0.90) Contribution of each group to each component of the MFA > res.mfa$group$contrib Dim.1 Dim.2 Dim.3 CGH 45.8 93.3 78.1 expr 54.2 6.7 21.9 

### • Similar contribution of the 2 groups to the rst dimension • Second dimension only due to CGH
51 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## The RV coecient X j ( I × K j ) 

### and X

m( I × K m ) 

### not directly comparable W j ( I × I ) 

### = X

j

### X0 j 

### and W

m( I × I ) 

### = X

m

### X0 m 

### can be compared Inner product matrices = relative position of the individuals Covariance between two groups:

< W j 

### , W m 

### > = X k ∈ K j 

### X l ∈ K m 

### cov2 

### (x

. k 

### , x
. l 

### ) Correlation between two groups: RV ( K j 

### , K m 

### ) = < W

j 

### , W m 

### > k W j 

### k k W m 

### k 0 ≤ RV ≤ 1 RV = 0: variables of K j 

### are uncorrelated with variables of K m 

### RV = 1: the two clouds of points are homothetic 52 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Partial analyses

• Comparison of the groups through the individuals ⇒ Comparison of the typologies provided by each group in a common space ⇒ Are there individuals very particular with respect to one group?

• Comparison of the separate PCA

53 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Projection of partial points

54 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Partial points

55 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Partial points

56 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Partial points

56 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the partial points

• 

### an individual is at the barycentre of its partial points • 

### an individual is all the more "homogeneous" that its superposed representations are close (res.mfa$ind$within.inertia) 57 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the partial components Do the separate analyses give similar dimensions as MFA?

58 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the partial components

• 

### The rst dimension of each group is well projected • 

### CGH has same dimensions as MFA

59 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the partial components

Separate PCA maps that can be compared to the MFA map slide 7 59 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Use of biological knowledge Genes can be grouped by gene ontology (GO) biological process

60 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Use of biological knowledge

• 

### Biological processes considered as supplementary groups of variables

61 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Use of biological knowledge

Many biological processes induce the same structure on the individuals than MFA

62 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Back to the wine example!

Objectives: • 

### How are the products described by the panels? • 

### Do the panels describe the products in a same way? Is there a specic description done by one panel? 63 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Practice with R

1 

### Dene groups of active and supplementary variables 2 

### Scale or not the variables 3 

### Perform MFA 4 

### Choose the number of dimensions to interpret 5 

### Simultaneously interpret the individuals and variables graphs 6 

### Study the groups of variables 7 

### Study the partial representations 8 

### Use indicators to enrich the interpretation

64 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Practice with R library(FactoMineR) Expert <- read.table("http://factominer.free.fr/useR2010/Expert_wine.csv", header=TRUE, sep=";", row.names=1) Consu <- read.table(".../Consumer_wine.csv",header=T,sep=";",row.names=1) Stud <- read.table(".../Student_wine.csv",header=T,sep=";",row.names=1) Pref <- read.table(".../Pref_wine.csv",header=T,sep=";",row.names=1) palette(c("black","red","blue","orange","darkgreen","maroon","darkviolet")) complet <- cbind.data.frame(Expert[,1:28],Consu[,2:16],Stud[,2:16],Pref) res.mfa <- MFA(complet,group=c(1,27,15,15,60),type=c("n",rep("s",4)), num.group.sup=c(1,5),graph=FALSE, name.group=c("Label","Expert","Consumer","Student","Preference")) plot(res.mfa,choix="group",palette=palette()) plot(res.mfa,choix="var",invisible="sup",hab="group",palette=palette()) plot(res.mfa,choix="var",invisible="actif",lab.var=FALSE,palette=palette()) plot(res.mfa,choix="ind",partial="all",habillage="group",palette=palette()) plot(res.mfa,choix="axes",habillage="group",palette=palette()) dimdesc(res.mfa) write.infile(res.pca,file="my_FactoMineR_results.csv") #to export a list 65 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the individuals

• 

### The two labels are well separated • 

### Vouvray are sensorially more dierent • 

### Several groups of wines, ...

66 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the active variables

67 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the active variables

67 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the groups

• 

### 2 groups are all the more close that they induce the same structure • 

### The 1st dimension is common to all the panels • 

### 2nd dimension mainly due to the experts • 

### Preference linked to sensory description

68 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the partial points

69 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Representation of the partial dimensions

• 

### The two rst dimensions of each group are well projected • 

### Consumer has same dimensions as MFA

70 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Representation of supplementary continuous variables

Preferences are linked to sensory description The favourite wine is Vouvray Aubussière Silex 71 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Helps to interpret

• 

### Contribution of each group of variables to each component of the MFA > res.mfa$group$contrib

Dim.1 Dim.2 Dim.3 Expert 30.5 46.0 33.7 Consumer 33.2 23.1 31.2 Student 36.3 30.9 35.1 • Similar contribution of the 3 groups to the rst dimension • Second dimension mainly due to the expert
• 

### Correlation between the global cloud and each partial cloud > res.mfa$group$correlation

Dim.1 Dim.2 Dim.3 Expert 0.95 0.95 0.96 Consumer 0.95 0.83 0.87 Student 0.99 0.99 0.84 

### First components are highly linked to the 3 groups: the 3 clouds of points are nearly homothetic
72 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Similarity measures between groups > res.mfa$group$Lg

Expert Consumer Student Preference Label MFA Expert 1.45 0.94 1.17 1.01 0.89 1.33 Consumer 0.94 1.25 1.04 1.11 0.28 1.21 Student 1.17 1.04 1.29 1.03 0.62 1.31 Preference 1.01 1.11 1.03 1.47 0.37 1.18 Label 0.89 0.28 0.62 0.37 1.00 0.67 MFA 1.33 1.21 1.31 1.18 0.67 1.44 > res.mfa$group$RV

Expert Consumer Student Preference Label MFA Expert 1.00 0.70 0.85 0.69 0.74 0.92 Consumer 0.70 1.00 0.82 0.82 0.25 0.90 Student 0.85 0.82 1.00 0.75 0.55 0.96 Preference 0.69 0.82 0.75 1.00 0.31 0.81 Label 0.74 0.25 0.55 0.31 1.00 0.56 MFA 0.92 0.90 0.96 0.81 0.56 1.00 • 

### Expert gives a richer description (Lg 

### greater) • 

### Groups Student and Expert are linked (RV = 0.85) • 

### Group Student is the closest to the overall (RV = 0.96) 73 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## To go further

• 

### Mixed data: MFA with 1 group = 1 variable if there are only continuous variables, PCA is recovered; if there are only categorical variables, MCA is recovered a specic function: AFDM • 

### MFA used for methodological purposes: • comparison of coding (continuous or categorical) • comparison between preprocessing (standardized PCA and unstandardized PCA) • comparison of results from dierent analyses • 

### Hierarchical Multiple Factor Analysis Takes into account a hierarchy on the variables: variables are grouped and subgrouped (like in questionnaires structured in topics and subtopics)
74 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

# Clustering and Principal Component

Methods

1 

### Clustering Methods 2 

### Principal Components Methods as a Preprocessing Step 3 

### Graphical Complementarity

75 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Unsupervised classication

• 

### Data set: table individuals × variables (or a distance matrix) • 

### Objective: to produce homogeneous groups of individuals (or groups of variables) • 

### Two kinds of clustering to dene two structures on individuals: hierarchy or partition

76 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Hierarchical Clustering Principle: sequentially agglomerate (clusters of) individuals using • 

### a distance between individuals: City block, Euclidean • 

### an agglomerative criterion: single linkage, complete linkage, average linkage, Ward's criterion
Representation with a dendrogram
⇒ Eulidean distance is used in principal component methods ⇒ Ward's criterion is based on multidimensional variance (inertia) which is the core of principal component methods 77 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Ascending Hierarchical Clustering AHC algorithm: • 

### Compute the Euclidean distance matrix ( I × I ) • 

### Consider each individual as a cluster • 

### Merge the two clusters A and B which are the closest with respect to the Ward's criterion:

∆ ward 

### (A , B ) = I A 

### IB 

### I A 

### + IB 

### d2 

### (μA 

### , μ B 

### ) with d the Euclidean distance, μA 

### the barycentre and I A 

### the cardinality of the set A • 

### Repeat until the number of clusters is equal to one
78 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Ward's criterion

• 

### Individuals can be represented by a cloud of points in R K • 

### Total inertia = multidimensional variance With Q groups of individuals, inertia can be decomposed as: K 

### X k = 1 Q 

### X q= 1 I

q 

### X i= 1 

### (x iqk 

### −  ̄ x k 

### ) 2 

### = K 

### X k = 1 Q 

### X q= 1 

### I q 

### (  ̄ x qk 

### −  ̄ x k 

### ) 2 

### + K 

### X k = 1 Q 

### X q= 1 I

q 

### X i= 1 

### (x iqk 

### −  ̄ x qk 

### ) 2 

### Total inertia = Between inertia + Within inertia

79 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Ward's criterion

⇒ Ward minimizes the increasing of within inertia 80 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## K-means algorithm

1 

### Choose Q points at random (the barycentre) 2 

### Aect the points to the closest barycentre 3 

### Compute the new barycentre 4 

### Iterate 2 and 3 until convergence

81 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## PCA as a preprocessing With continuous variables: ⇒ AHC and k-means onto the raw data ⇒ AHC or k-means onto principal components PCA transforms the raw variables into orthogonal principal components F

. 1 

### , ..., F

. K 

### with decreasing variance λ 1 

### ≥ λ 2 

### ≥ ...λ K



### ⇒ Keeping the rst components makes the clustering more robust ⇒ But, how many components do you keep to denoise? 82 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## MCA as a preprocessing Clustering on categorical variables: which distance to use? • 

### with two categories: Jaccard index, Dice's coecient, simple match, etc. Indices well-tted for presence/absence data • 

### with more than 2 categories: use for example the χ2

### -distance Using the χ2

### -distance ⇔ computing distances from all the principal components obtained from MCA In practice, MCA is used as a preprocessing in order to • 

### transform categorical variables in continuous ones • 

### delete the last dimensions to make the clustering more robust
83 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## MFA as a preprocessing

MFA balances the inuence of the groups when computing distances between individuals

d2 

### ( i , i0 

### ) = J



### X j= 1 

### 1 pλ j K j 

### X k = 1 

### (x ik 

### − x i0 k 

### ) 2 

### AHC or k-means onto the rst principal components ( F

. 1 

### , ..., F

. Q 

### ) obtained from MFA allows to • take into account the groups structure in the clustering • make the clustering more robust by deleting the last dimensions 84 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Back to the wine data! AHC onto the rst 5 principal components from MFA

Individuals are sorted according to their coordinate F

. 1 85 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Why sorting the tree? X <- c(6,7,2,0,3,15,11,12) names(X) <- X library(cluster) par(mfrow=c(1,2)) plot(as.dendrogram(agnes(X))) plot(as.dendrogram(agnes(sort(X))))

86 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Partition from the tree An empirical number of clusters is suggested (min q W q − W q+ 1 W q − 1 − W

q 

### )

87 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Partition from the tree An empirical number of clusters is suggested (min q W q − W q+ 1 W q − 1 − W

q 

### )

87 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Hierarchical tree on the principal component map

Hierarchical tree gives an idea of the other dimensions 88 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Partition on the principal component map

Continuous view (principal components) and discontinuous (clusters) 89 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Cluster description by variables v.test =  ̄

x q 

### −  ̄ x r s2 I q 

###  I

− I

q I

− 1 

###  ∼ N( 0 , 1 ) H 0 

### :  ̄ x q 

### =  ̄ x with  ̄ x q 

### the mean of variable x in cluster q ,  ̄ x (s) the mean (standard deviation) of the variable x in the data set, I q 

### the cardinal of cluster q $desc.var$quanti$`2`

v.test Mean in Overall sd in Overall p.value category mean category sd O.passion_C 2.58 6.17 4.61 0.79 1.18 0.01 O.citrus 2.50 5.40 3.66 0.22 1.37 0.01 O.passion_S 2.45 5.69 4.18 0.54 1.20 0.01

.... Typicity -2.42 1.36 3.91 0.72 2.07 0.02 O.candied.fruit -2.44 0.78 2.58 0.16 1.45 0.01 O.alcohol_S -2.48 3.98 4.33 0.13 0.28 0.01 Surface.feeling -2.52 2.63 3.62 0.12 0.77 0.01 90 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Cluster description

• by the principal components (individuals coordinates) : same description than for continuous variables $desc.axes$quanti$`2` v.test Mean in Overall sd in Overall p.value category mean category sd Dim.2 2.20 1.39 7.77e-17 0.253 1.24 0.0276 

### • by categorical variables : chi-square and hypergeometric test
⇒ Active and supplementary elements are used ⇒ Only signicant results are presented
91 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Cluster description by individuals • parangon: the closest individuals to the barycentre of the cluster min i ∈ q 

### d (x

i . 

### , μ q 

### ) with μ q 

### the barycentre of cluster q • specic individuals: the furthest individuals to the barycentres of the other clusters (the individuals sorted according to their distance from the highest to the smallest to the closest barycentre) max i ∈ q 

### min q 0 6 = q 

### d (x i . 

### , μ q 0 

### ) desc.ind$para cluster: 2 S Renaudie S Trotignon S Michaud 0.1002890 0.3101154 0.3640145

------------------------------------------ desc.ind$dist cluster: 2 S Trotignon S Renaudie S Michaud 1.934103 1.687849 1.265386

------------------------------------------ 92 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Complementarity between hierarchical clustering and

partitioning

• 

### Partitioning after AHC: the k-means algorithm is initialized from the barycentres of the partition obtained from the tree • consolidate the partition • loss of the hierarchy

• 

### AHC with many individuals: time-consuming ⇒ partitioning before AHC • compute k-means with approximately 100 clusters • AHC on the weighted barycentres obtained from the k-means ⇒ top of the tree is approximately the same

93 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Practice with R res.hcpc <- HCPC(res.mfa) ##### Example of clustering on categorical data data(tea) res.mca <- MCA(tea,quanti.sup=19,quali.sup=20:36) plot(res.mca,invisible=c("var","quali.sup","quanti.sup"),cex=0.7) plot(res.mca,invisible=c("ind","quali.sup","quanti.sup"),cex=0.8) plot(res.mca,invisible=c("quali.sup","quanti.sup"),cex=0.8) dimdesc(res.mca) res.mca <- MCA(tea,quanti.sup=19,quali.sup=20:36, ncp=10) res.hcpc <- HCPC(res.mca)

94 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## CARME conference International conference on Correspondence Analysis and

Related MEthods Agrocampus Rennes (France), February 8-11, 2011 R tutorials for corresp. ana. and related methods of visualization: • S. Dray: multivariate analysis of ecological data with ade4 • O. Nenadi¢ & M. Greenacre: correspondence analysis with ca • S. Lê: from one to multiple data tables with FactoMineR • J. de Leeuw & P. Mair: multidimensional scaling using majorisation with smacof 

### Invited speakers : Monica Bécue, Cajo ter Braak, Jan de Leeuw, Stéphane Dray, Michael Friendly, Patrick Groenen, Pieter Kroonenberg 95 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## Bibliography • Escoer B. & Pagès J. (1994). Multiple factor analysis (AFMULT package). Computational Statistics and Data Analysis, 121-140. • Greenacre M. & Blasius J. (2006). Multiple Correspondence Analysis and related methods. Chapman & Hall/CRC. • Husson F., Lê S. & Pagès J. (2010). Exploratory Multivariate Analysis by Example Using R. Chapman & Hall. • Jollie I. (2002). Principal Component Analysis. Springer. 2nd edn. • Lebart L., Morineau A. & Warwick K. (1984). Multivariate descriptive statistical analysis. Wiley, New-York. • Le Roux B. & Rouanet H. (2004). Geometric Data Analysis, From Correspondence Analysis to Structured Data Analysis
. Dordrecht: Kluwer.

96 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods



## Packages' bibliography http://cran.r-project.org/web/views/Multivariate.html http://cran.r-project.org/web/views/Cluster.html
• ade4 package: data analysis functions to analyse Ecological and Environmental data in the framework of Euclidean Exploratory methods http://pbil.univ-lyon1.fr/ADE-4 • ca package (Greenacre and Nenadic) deals with simple, multiple and joint correspondence analysis • cluster package: basic and hierarchical clustering • dynGraph package: visualization software to explore interactively graphical outputs provided by multidimensional methods http://dyngraph.free.fr • FactoMineR package http://factominer.free.fr • hopach package: builds hierarchical tree of clusters • missMDA package: imputes missing values with multivariate data analysis methods 97 / 98Principal Component Analysis Multiple Factor Analysis Clustering and Principal Component Methods 

## FactoMineR A website with documentation, examples, data sets: http://factominer.free.fr How to install the Rcmdr menu: copy and paste the following line of code in a R session source("http://factominer.free.fr/install-facto.r") 

### A book: Husson F., Lê S. & Pagès J. (2010). Exploratory Multivariate Analysis by Example Using R. Chapman & Hall.

98 / 98