3 Weeks Beginners Guide to Ace Data Science Interview: #Day 14
By: Vinay Vikram
Tags: Interview Questions, Machine Learning, Interview, Python, Data Science

Interviewer: Do you know the math which works in the back of LDA and PCA.

My reaction...

A question so many interviewers asked to check your in-depth Mathematics Knowledge

Question6: While performing PCA and LDA we work a lot on eigenvector and eigenvalues. So could you explain what are these mathematical terms & what is their physical significance?

Answer:

If you consider a matrix to be a linear operator that maps one space into another, then an eigenvector is a direction in the first space which is preserved under the mapping in the second space. The eigenvalue is simply how much a vector in that direction is stretched.

In linear algebra, an eigenvector or characteristic vector of a square matrix is a vector that does not change its direction under the associated linear transformation. In other words — if v is a vector that is not zero, then it is an eigenvector of a square matrix A if Av is a scalar multiple of v. This condition could be written as the equation

Av =λv

where λ is a number (also called a scalar) known as the eigenvalue or characteristic value associated with the eigenvector v→.

In this shear mapping, the red arrow changes direction but the blue arrow does not. The blue arrow is an eigenvector of this shear mapping because it doesn’t change direction,

To have deeper intuition go through following links:

Question 7: How to find the optimum number of Principal components in PCA?

Answer:

To find the optimum number of components we create a plot of cumulative Summation of the Explained Variance against the number of features and then we select the appropriate number of features.

This plot tells us that selecting 5 components we can preserve something around 98.8% or 99% of the total variance of the data. It makes sense, we’ll not use 100% of our variance, because it denotes all components, and we want only the principal ones.

With this information in our hands, we can implement the PCA for 5 best components.

Question 8: Before we perform PCA, It’s advisable to perform feature scaling. Why So?

Answer:

To perform a reasonable covariance analysis we need to normalize this data, putting all attributes in the same unit of measurement (the range between 0 and 1), it will improve the maximization of the variance for each component that our PCA needs to perform its matrix operations in the best way.

Question 9: Which of the following gives the difference(s) between the logistic regression and LDA?

Answer:

(1)If the classes are well separated, the parameter estimates for logistic regression can be unstable.

(2) If the sample size is small and distribution of features are normal for each class. In such cases, linear discriminant analysis is more stable than logistic regression.

A. 1

B. 2

C. 1 and 2

D. None of these

Answer: C

Question 10: Is rotation(having orthogonal components)necessary in PCA? If yes, Why? What will happen if you don’t rotate the components?

Answer:

Yes, rotation (orthogonal) is necessary because it maximizes the difference between variance captured by the component. This makes the components easier to interpret. Not to forget, that’s the motive of doing PCA where, we aim to select fewer components (than features) which can explain the maximum variance in the data set. By doing rotation, the relative location of the components doesn’t change, it only changes the actual coordinates of the points.

If we don’t rotate the components, the effect of PCA will diminish and we’ll have to select more number of components to explain variance in the data set.

Thanks for reading!