Variance inflation factor
In statistics, the variance inflation factor (VIF) is the ratio (quotient) of the variance of estimating some parameter in a model that includes multiple other terms (parameters) by the variance of a model constructed using only one term. It quantifies the severity of multicollinearity in an ordinary least squares regression analysis. It provides an index that measures how much the variance (the square of the estimate's standard deviation) of an estimated regression coefficient is increased because of collinearity. Cuthbert Daniel claims to have invented the concept behind the variance inflation factor, but did not come up  with the name. Definition. Consider the following linear model with "k" independent variables: The standard error of the estimate of "β" is the square root of the "j" + 1 diagonal element of "s"("X"′"X"), where "s" is the root mean squared error (RMSE) (note that RMSE is a consistent estimator of the true variance of the error term, formula_1); "X" is the regression design matrix — a matrix such that "X" is the value of the "j" independent variable for the "i" case or observation, and such that "X", the predictor vector associated with the intercept term, equals 1 for all  i. It turns out that the square of this standard error, the estimated variance of the estimate of "β", can be equivalently expressed as: where "R" is the multiple "R" for the regression of "X" on the other covariates (a regression that does not involve the response variable "Y"). This identity separates the influences of several distinct factors on the variance of the coefficient estimate: The remaining term, 1 / (1 − "R") is the VIF. It reflects all other factors that influence the uncertainty in the coefficient estimates. The VIF equals 1 when the vector "X" is orthogonal to each  column of the design matrix for the regression of "X" on the other covariates. By contrast, the VIF is greater than 1 when the vector "X" is not orthogonal to all columns of the design matrix for the regression of "X" on the other covariates. Finally, note that the VIF is invariant to the scaling of the variables (that is, we could scale each variable "X" by a constant "c" without changing the VIF). Now let formula_5, and without losing generality, we reorder the columns of "X" to set the first column to be formula_6 By using Schur complement, the  element in the first row and first column in formula_9 is, Then we have, Here formula_12 is the coefficient of regression of dependent variable formula_13 over covariate formula_14. formula_15 is the corresponding residual sum of squares. Calculation and analysis. We can calculate "k" different VIFs (one for each "X") in three steps: Step one. First we run an ordinary least square regression that has "X" as a function of all the other explanatory variables in the first equation. If "i" = 1, for example, equation would be where formula_17 is a constant and "e" is the error term. Step two . Then, calculate the VIF factor for formula_18 with the following formula : where "R" is the coefficient of determination of the regression equation in step one, with formula_20 on the left hand side, and all other predictor variables (all the other X variables) on the right hand side. Step three. Analyze the magnitude of multicollinearity by considering the size of the formula_21. A rule of thumb is that if formula_22 then multicollinearity is high (a cutoff of 5 is also commonly used). Some software instead calculates the tolerance which is just the reciprocal of the VIF. The choice of which to  use is a matter of personal preference. Interpretation. The square root of the variance inflation factor indicates how much larger the standard error increases compared to if that variable had 0 correlation to other predictor variables in the model. Example If the variance inflation factor of a predictor variable were 5.27 (√5.27 = 2.3), this means that the standard error for the coefficient of that predictor variable is 2.3 times larger than if that predictor variable had 0 correlation with the other predictor variables.