Exploratory Factor Analysis in R. Learning by doing | by Anh T. Dang | Towards Data Science

Factor analysis is a statistical method used to search for some unobserved variables called factors from observed variables called factors.
This beginning of the method was named exploratory factor analysis (EFA).
Another variation of factor analysis is confirmatory factor analysis (CFA) will not be explored in this article.
Factor analysis is used in many areas of statistical analysis such as marketing, social sciences, psychology, and so on.
The libraries that we’ll need are as follows: psych, corrplot, psynch and so on.
library(psych)library(corrplot)library("psych")library(ggplot2)library(car)
We first need to fetch sample data from the server. This is a raw data set, so each row row represents a person’s survery.
# Dataseturl <- “https://raw.githubusercontent.com/housecricket/data/main/efa/sample1.csv"data_survey <- read.csv(url, sep = “,”)
We also look at the dataset before we run any analysis.
describe(data_survey)
We use the dim function to retrieve the dimension of the dataset.
dim(data_survey)
In our data frame, we have an ID variable in the first column. So, we can use a -1 in the column index to remove the first column and save our data to a new object.
dat <- data_survey[ , -1] head(dat)
We also should take a look at the correlations among our variables to determine if factor analysis is appropriate.
datamatrix <- cor(dat[,c(-13)])corrplot(datamatrix, method="number")
X <- data[,-c(13)]Y <- data[,13]
The Kaiser-Meyer-Olkin (KMO) used to measure sampling adequacy is a better measure of factorability.
KMO(r=cor(X))
According to Kaiser’s (1974) guidelines, a suggested cutoff for determining the factorability of the sample data is KMO ≥ 60. The total KMO is 0.83, indicating that, based on this test, we can probably conduct a factor analysis.
cortest.bartlett(X)
Small values (8.84e-290 < 0.05) of the significance level indicate that a factor analysis may be useful with our data.
det(cor(X)
We have a positive determinant, which means the factor analysis will probably run.
library(ggplot2)fafitfree <- fa(dat,nfactors = ncol(X), rotate = "none")n_factors <- length(fafitfree$e.values)scree     <- data.frame(  Factor_n =  as.factor(1:n_factors),   Eigenvalue = fafitfree$e.values)ggplot(scree, aes(x = Factor_n, y = Eigenvalue, group = 1)) +   geom_point() + geom_line() +  xlab("Number of factors") +  ylab("Initial eigenvalue") +  labs( title = "Scree Plot",         subtitle = "(Based on the unreduced correlation matrix)")
We can use the parallel() function from the nFactors package (Raiche & Magis, 2020) to perform a parallel analysis.
parallel <- fa.parallel(X)
## Parallel analysis suggests that the number of factors =  4 and the number of components =  3
fa.none <- fa(r=X,  nfactors = 4,  # covar = FALSE, SMC = TRUE, fm=”pa”, # type of factor analysis we want to use (“pa” is principal axis factoring) max.iter=100, # (50 is the default, but we have changed it to 100 rotate=”varimax”) # none rotationprint(fa.none)
factanal.none <- factanal(X, factors=4, scores = c("regression"), rotation = "varimax")print(factanal.none)
fa.diagram(fa.none)
head(fa.var$scores)
regdata <- cbind(dat[“QD”], fa.var$scores)#Labeling the datanames(regdata) <- c(“QD”, “F1”, “F2”, “F3”, “F4”)head(regdata)
#Splitting the data 70:30#Random number generator, set seed.set.seed(100)indices= sample(1:nrow(regdata), 0.7*nrow(regdata))train=regdata[indices,]test = regdata[-indices,]
model.fa.score = lm(Satisfaction~., train)summary(model.fa.score)
Our model equation can be written as: Y = 3.55309 + 0.72628 x F1 + 0.29138 x F2 + 0.06935 x F3 + 0.62753 x F4
vif(model.fa.score)
#Model Performance metrics:pred_test <- predict(model.fa.score, newdata = test, type = “response”)pred_testtest$QD_Predicted <- pred_testhead(test[c(“QD”,”QD_Predicted”)], 10)
Today, we reviewed the preliminary steps to factor analysis including examining the data and the assumptions required for factor analysis and how to determine the number of factors to retain.