# Revision: Variance Inflation in Regression

Authors: D. R. Jensen; D. E. Ramirez
Journal: Advances in Decision Sciences
 (2013)

---

## Abstract

Variance Inflation Factors (VIFs) are reexamined as conditioning diagnostics for models with intercept, with and without centering regressors to their means as oft debated. Conventional VIFs, both centered and uncentered, are flawed. To rectify matters, two types of orthogonality are noted: vector-space orthogonality and uncorrelated centered regressors. The key to our approach lies in feasible Reference models encoding orthogonalities of these types. For models with intercept it is found that (i)
uncentered VIFs are not ratios of variances as claimed, owing to infeasible Reference models; (ii) instead they supply informative angles between subspaces of regressors; (iii) centered VIFs are incomplete if not misleading, masking collinearity of regressors with
the intercept; and (iv)variance deflation may occur, where ill-conditioned data yield smaller variances than their orthogonal surrogates. Conventional VIFs have all regressors linked, or none, often untenable in practice. Beyond these, our models enable the unlinking of regressors that can be unlinked, while preserving dependence among those intrinsically linked. Moreover, known collinearity indices are extended to encompass angles between subspaces of regressors. To reaccess ill-conditioned data, we consider case studies ranging from elementary examples to data from the literature.


---

## Body

## 1. Introduction

Values for residualsRSj=∥xj-Pjxj∥2, for  ∥Xj∥2, for κuj2=VIFu(βj^), for (1-1/κuj2)1/2, and for θj, for j=0,1,2.Given{Y=Xβ+ϵ} of full rank with zero-mean, uncorrelated and homoscedastic errors, the p equations {X′Xβ=X′Y} yield the OLS estimators β^ for β=[β1,…,βp]′ as unbiased with dispersion matrix V(β^)=σ2V and V=[vij]=(X′X)-1.   Ill-conditioning, as near dependent columns of X, exerts profound and interlinked consequences, causing “crucial elements of X′X to be large and unstable,” “creating inflated variances”; estimates excessive in magnitude, irregular in sign, and “very sensitive to small changes in X”; and unstable algorithms having “degraded numerical accuracy.” See [1–3] for example.Ill-conditioning diagnostics include thecondition numberc1(X′X), the ratio of its largest to smallest eigenvalues, and the Variance Inflation Factors{VIF(β^j)=ujjvjj;1≤j≤p} with U=X′X,  that is, ratios of actual (vjj) to “ideal” (1/ujj) variances, had the columns of X been orthogonal. On scaling the latter to unit lengths and ordering {VIF(β^j)=vjj;1≤j≤p} as {V1≥V2≥⋯≥Vp},  V1 is identified in [4] as “the best single measure of the conditioning of the data.” In addition, the bounds V1≤c1(X′X)≤p(V1+⋯+Vp) of [5] apply also in stepwise regression as in [6–9].Users deserve to be apprised not only that data are ill conditioned, but also about workings of the diagnostics themselves. Accordingly, we undertake here to rectify long-standing misconceptions in the use and properties ofVIFs, necessarily retracing through several decades to their beginnings.To choose a model, with or without intercept, is substantive, is specific to each experimental paradigm, and is beyond the scope of the present study. Whatever the choice, fixed in advance in a particular setting, these models follow on specializingp=k+1, then p=k, respectively, as M0:{Y=X0θ+ϵ} with intercept, and Mw:{Y=Xβ+ϵ} without, where X0=[1n,X]; X=[X1,X2,…,Xk] comprise vectors of regressors; and  θ'=[β0,β'] with β0 as intercept and β′=[β1,…,βk] as slopes. The VIFs as defined are essentially undisputed for models in Mw, as noted in [10], serving to gage effects of nonorthogonal regressors as ratios of variances. In contrast, a yet unresolved debate surrounds the choice of conditioning diagnostics for models in M0, namely, between uncentered regressors giving VIFus for [β^0,β^1,…,β^k] and regressors Xi   centered to their means, giving VIFcs for [β^1,…,β^k]. Specifically, {VIFu(β^j)=ujjvjj;j=0,1,…,k} on taking U=X0′X0 and V=(X0′X0)-1. In contrast, letting  R=[Rij] be the correlation matrix for β^=[β^1,…,β^k]′ and R-1=[Rij] its inverse, then centered versions are {VIFc(β^j)=Rjj;1≤j≤k} for slopes only.It is seen here that (i) these differ profoundly in regard to their respective concepts of orthogonality; (ii) objectives and meanings differ accordingly; (iii) sharp divisions trace to muddling these concepts; and (iv) this distinction assumes a pivotal role here. Over time a large body of widely held beliefs, conjectures, intrinsic propositions, and conventional wisdom has accumulated, much from flawed logic, some to be dispelled here. The key to our studies is thatVIFs, to be meaningful, must compare actual variances to those of an “ideal” second-moment matrix as reference, the latter to embody the conjectured type of orthogonality. This differs between centered and uncentered diagnostics and for both types requires the reference matrix to be feasible. An outline follows.Our undertaking consists essentially of four parts. The first is a literature survey of some essentials of ill-conditioning, to include the divide between centered and uncentered diagnostics and conventionalVIFs. The structure of orthogonality is examined next. Anomalies in usage, meaning, and interpretation of conventional VIFs are exposed analytically and through elementary and transparent case studies. Long-standing but ostensible foundations in turn are reassessed and corrected through the construction of “Reference models.” These are moment arrays constrained to encode orthogonalities of the types considered. Neither array returns the conventional  VIFus nor VIFcs. Direct rules for finding the amended Reference models are given, preempting the need for constrained numerical algorithms. Finally, studies of ill-conditioned data from the literature are reexamined in light of these findings.


## 2. Preliminaries

### 2.1. Notation

Designate byℝp the Euclidean p-space. Matrices and vectors are set in bold type; the transpose and inverse of A are A′ and A-1; and A(ij) refers on occasion to the (i,j) element of A. Special arrays are the identity Ip; the unit vector 1p = [1,1,…,1]′∈ℝp; the diagonal matrix Da=Diag(a1,…,ap); and Bn = (In-n-11n1n′), as idempotent of rank n-1. The Frobenius norm of x∈ℝp is ∥x∥ = (x′x)1/2. For X of order (n×p) and rank p≤n, a generalized inverse is designated as X†; its ordered singular values as σ(X)={ξ1≥ξ2≥⋯≥ξp>0}; and by 𝒮p(X)⊂ℝn the subspace of ℝn spanned by the columns of X. By accepted convention its condition number is c2(X)=ξ1/ξp, specifically, c2(X)=[c1(X′X)]1/2. For our model {Y=Xβ+ϵ}, with dispersion V(ϵ)=σ2In, we take σ2=1.0 unless stated otherwise, since variance ratios are scale invariant.


### 2.2. Case Study 1: A First Look

That anomalies pervade conventionalVIFus and VIFcs may be seen as follows. Given M0:{Yi=β0+β1X1+β2X2+ϵi} the design X0=[15,X1,X2] of order (5×3),U=X0′X0, and its inverse V=(X0′X0)-1 as in
(1)X0′=[1111100.50.511-11100],U=[53132.51113],V=[0.72222-0.888890.05556-0.888891.55556-0.222220.05556-0.222220.38889].
Conventional centered and uncentered VIFs are
(2)[VIFc(β^1),VIFc(β^2)]=[1.0889,1.0889],(3)[VIFu(β^0),VIFu(β^1),VIFu(β^2)]=[3.6111,3.8889,1.1667],
respectively, the former for slopes only and the latter taking reciprocals of the diagonals of X0′X0 as reference.A Critique. The following points are basic. Note first that model (1) is nonorthogonal in both the centered and uncentered regressors.Remark 1.
TheVIFus are not ratios of variances and thus fail to gage relative increases in variances owing to nonorthogonal columns of  X0. This follows since the first row and column of the second-moment matrix X0′X0=U are fixed and nonzero by design, so that taking X0′X0 to be diagonal as reference cannot be feasible.Remark1 runs contrary to assertions throughout the literature. In consequence, for models in M0 the mainstay VIFus in recent vogue are largely devoid of meaning. Subsequently these are identified instead with angles quantifying degrees of multicollinearity among the regressors.On the other hand, feasible Reference modelsfor all parameters, as identified later for centered and uncentered data in Definition 13, Section 4.2, give
(4)[VFc(β^0),VFc(β^1),VFc(β^2)]=[0.9913,1.0889,1.0889],(5)[VFv(β^0),VFv(β^1),VFv(β^2)]=[0.7704,0.8889,0.8889]in lieu of conventional VIFcs and VIFus, respectively. The former comprise corrected VIFcs, extended to include the intercept. Both sets in fact are genuine variance inflation factors, as ratios of variances in the model (1), relative to those in Reference models feasible for centered and for uncentered regressors, respectively.This example flagrantly contravenes conventional wisdom: (i) variances for slopes are inflated in (4), but for the intercept deflated, in comparison with the feasible centered reference. Specifically, β0 is estimated here with greater efficiency VFc(β^0)=0.9913 in the initial design (1), despite nonorthogonality of its centered regressors. (ii) Variances are uniformly smaller in the model (1) than in its feasible uncentered reference from (5), thus exhibiting Variance Deflation, despite nonorthogonality of the uncentered regressors. A full explication of the anatomy of this study appears later. In support, we next examine technical details needed in subsequent developments.


### 2.3. Types of Orthogonality

The ongoing divergence between centered and uncentered diagnostics traces in part to meaning ascribed toorthogonality. Specifically, the orthogonality of columns of  X in Mw refers unambiguously to the vector-space concept Xi⊥Xj, that is,Xi′Xj=0, as does the notion of collinearity of regressors with the constant vector in M0. We refer to this as V-orthogonality, in short V⊥. In contrast, nonorthogonality in M0 often refers instead to the statistical concept of correlation among columns of X when scaled and centered to their means. We refer to its negation as  C-orthogonality, or C⊥. Distinguishing between these notions is fundamental, as confusion otherwise is evident. For example, it is asserted in [11, p.125] that “the simple correlation coefficient rij does measure linear dependency between xi and xj  in the data.”


### 2.4. The ModelsM0

ConsiderM0:{Y=X0θ+ϵ}, X0=[1n,X], θ′=[β0,β′], together with
(6)X0′X0=[nnX-′nX-X′X],(X0′X0)-1=[c00c01c01′G],
where nX-′=1n′X, c00=[n-n2X-′(X′X)-1X-]-1, C=(X′X-nX-X-′) = X′BnX, and G=C-1 from block-partitioned inverses. In later sections, we often will denote the mean-centering matrix nX-X-′ by M. In particular, the centered form {X→Z=BnX} arises exclusively in models with intercept, with or without reparametrizing (β0,β)→(α,β) in the mean-centered regressors, where α = β0+β1X-1+⋯+βkX-k. Scaling Z to unit column lengths gives Z′Z in correlation form with unit diagonals.


## 2.1. Notation

Designate byℝp the Euclidean p-space. Matrices and vectors are set in bold type; the transpose and inverse of A are A′ and A-1; and A(ij) refers on occasion to the (i,j) element of A. Special arrays are the identity Ip; the unit vector 1p = [1,1,…,1]′∈ℝp; the diagonal matrix Da=Diag(a1,…,ap); and Bn = (In-n-11n1n′), as idempotent of rank n-1. The Frobenius norm of x∈ℝp is ∥x∥ = (x′x)1/2. For X of order (n×p) and rank p≤n, a generalized inverse is designated as X†; its ordered singular values as σ(X)={ξ1≥ξ2≥⋯≥ξp>0}; and by 𝒮p(X)⊂ℝn the subspace of ℝn spanned by the columns of X. By accepted convention its condition number is c2(X)=ξ1/ξp, specifically, c2(X)=[c1(X′X)]1/2. For our model {Y=Xβ+ϵ}, with dispersion V(ϵ)=σ2In, we take σ2=1.0 unless stated otherwise, since variance ratios are scale invariant.


## 2.2. Case Study 1: A First Look

That anomalies pervade conventionalVIFus and VIFcs may be seen as follows. Given M0:{Yi=β0+β1X1+β2X2+ϵi} the design X0=[15,X1,X2] of order (5×3),U=X0′X0, and its inverse V=(X0′X0)-1 as in
(1)X0′=[1111100.50.511-11100],U=[53132.51113],V=[0.72222-0.888890.05556-0.888891.55556-0.222220.05556-0.222220.38889].
Conventional centered and uncentered VIFs are
(2)[VIFc(β^1),VIFc(β^2)]=[1.0889,1.0889],(3)[VIFu(β^0),VIFu(β^1),VIFu(β^2)]=[3.6111,3.8889,1.1667],
respectively, the former for slopes only and the latter taking reciprocals of the diagonals of X0′X0 as reference.A Critique. The following points are basic. Note first that model (1) is nonorthogonal in both the centered and uncentered regressors.Remark 1.
TheVIFus are not ratios of variances and thus fail to gage relative increases in variances owing to nonorthogonal columns of  X0. This follows since the first row and column of the second-moment matrix X0′X0=U are fixed and nonzero by design, so that taking X0′X0 to be diagonal as reference cannot be feasible.Remark1 runs contrary to assertions throughout the literature. In consequence, for models in M0 the mainstay VIFus in recent vogue are largely devoid of meaning. Subsequently these are identified instead with angles quantifying degrees of multicollinearity among the regressors.On the other hand, feasible Reference modelsfor all parameters, as identified later for centered and uncentered data in Definition 13, Section 4.2, give
(4)[VFc(β^0),VFc(β^1),VFc(β^2)]=[0.9913,1.0889,1.0889],(5)[VFv(β^0),VFv(β^1),VFv(β^2)]=[0.7704,0.8889,0.8889]in lieu of conventional VIFcs and VIFus, respectively. The former comprise corrected VIFcs, extended to include the intercept. Both sets in fact are genuine variance inflation factors, as ratios of variances in the model (1), relative to those in Reference models feasible for centered and for uncentered regressors, respectively.This example flagrantly contravenes conventional wisdom: (i) variances for slopes are inflated in (4), but for the intercept deflated, in comparison with the feasible centered reference. Specifically, β0 is estimated here with greater efficiency VFc(β^0)=0.9913 in the initial design (1), despite nonorthogonality of its centered regressors. (ii) Variances are uniformly smaller in the model (1) than in its feasible uncentered reference from (5), thus exhibiting Variance Deflation, despite nonorthogonality of the uncentered regressors. A full explication of the anatomy of this study appears later. In support, we next examine technical details needed in subsequent developments.


## 2.3. Types of Orthogonality

The ongoing divergence between centered and uncentered diagnostics traces in part to meaning ascribed toorthogonality. Specifically, the orthogonality of columns of  X in Mw refers unambiguously to the vector-space concept Xi⊥Xj, that is,Xi′Xj=0, as does the notion of collinearity of regressors with the constant vector in M0. We refer to this as V-orthogonality, in short V⊥. In contrast, nonorthogonality in M0 often refers instead to the statistical concept of correlation among columns of X when scaled and centered to their means. We refer to its negation as  C-orthogonality, or C⊥. Distinguishing between these notions is fundamental, as confusion otherwise is evident. For example, it is asserted in [11, p.125] that “the simple correlation coefficient rij does measure linear dependency between xi and xj  in the data.”


## 2.4. The ModelsM0

ConsiderM0:{Y=X0θ+ϵ}, X0=[1n,X], θ′=[β0,β′], together with
(6)X0′X0=[nnX-′nX-X′X],(X0′X0)-1=[c00c01c01′G],
where nX-′=1n′X, c00=[n-n2X-′(X′X)-1X-]-1, C=(X′X-nX-X-′) = X′BnX, and G=C-1 from block-partitioned inverses. In later sections, we often will denote the mean-centering matrix nX-X-′ by M. In particular, the centered form {X→Z=BnX} arises exclusively in models with intercept, with or without reparametrizing (β0,β)→(α,β) in the mean-centered regressors, where α = β0+β1X-1+⋯+βkX-k. Scaling Z to unit column lengths gives Z′Z in correlation form with unit diagonals.


## 3. Historical Perspective

Our objective is an overdue revision of the tenets of variance inflation in regression. To provide context, we next survey extenuating issues from the literature. Direct quotations are intended not to subvert stances taken by the cited authors. Models inM0 are at issue since, as noted in [10], centered diagnostics have no place in Mw.


### 3.1. Background

Aspects of ill-conditioning span a considerable literature over many decades. Regarding{Y=Xβ+ϵ}, scaling columns of X to equal lengths approximately minimizes the condition number c2(X) [12, p.120] based on [13]. Nonetheless, c2(X) is cast in [9] as a blunt instrument for ill-conditioning, prompting the need for VIFs and other local constructs. Stewart [9] credits VIFs in concept to Daniel and in name to Marquardt.Ill-conditioning points beyondOLS in view of difficulties cited earlier. Remedies proffered in [14, 15] include transforming variables, adding new data, and deleting variable(s) after checking critical features of the reduced model. Other intended palliatives include Ridge and Partial Least Squares, as compared in [16]; Principal Components regression; Surrogate models as in [17]. All are intended to return reduced standard errors at the expense of bias. Moreover, Surrogate solutions more closely resemble those from an orthogonal system than Ridge [18]. Together the foregoing and other options comprise a substantial literature as collateral to, but apart from, the present study.


### 3.2. To Center

Advocacy for centering includes the following.(i)
VIFs often are defined categorically as the diagonals of the inverse of the correlation matrix of scaled and centered regressors; see [4, 9, 11, 19] for example. These are VIFcs, widely adopted without justification as default to the exclusion of VIFus.(ii)
It is asserted [4] that “centering removes the nonessential ill-conditioning, thus reducing the variance inflation in the coefficient estimates.”(iii)
Centering is advocated when predictor variables are far removed from origins on the basic data scales [10, 11].


### 3.3. Not to Center

Advocacy for uncentered diagnostics includes the following caveats from proponents of centering.(i)
Uncentered data should be examined only if an estimate of the intercept is of interest [9, 10, 20].(ii)
“If the domain of prediction includes the full range from the natural origin through the range of data, the collinearity diagnostics should not be mean centered” [10, p.84].Other issues against centering derive in part from numerical analysis and work by Belsley.(i)
Belsley [1] identifies c2(X) for a system {Y=Xβ+ϵ} as “the potential relative change in the LS solution b that can result from a small relative change in the data.”(ii)
These requirestructurally interpretable variables as “ones whose numerical values and (relative) differences derive meaning and interpretability from knowledge of the structure of the underlying ‘reality’ being modeled” [1, p.75].(iii)
“There is no such thing as ‘nonessential’ ill-conditioning,” and “mean-centering can remove from the data the information needed to assess conditioning correctly” [1, p.74].(iv)
“Collinearity with the intercept can quite generally corrupt the estimates ofall parameters in the model whether or not the intercept is itself of interest and whether or not the data have been centered” [21, p.90].(v)
An example [22, p.121] gives severely ill-conditioned data perfectly conditioned in centered form: “centering alters neither” inflated variances nor extremely sensitive parameter estimates in the basic data; moreover, “diagnosing the conditioning of the centered data (which are perfectly conditioned) would completely overlook this situation, whereas diagnostics based on the basic data would not.”(vi)
To continue from [22], ill-conditioning persists in the propagation of disturbances, in that “a 1 percent relative change in the Xi’s results in over a 40% relative change in the estimates,” despite perfect conditioning in centered form, and “knowledge of the effect of small relative changes in the centered data is not meaningful for assessing the sensitivity of the basic LS problem,” since relative changes and their meanings in centered and uncentered data often differ markedly.(vii)
Regarding choice of origin, “(1) the investigator must be able to pick an origin against which small relative changes can be appropriately assessed and (2) it is the data measured relative to this origin that are relevant to diagnosing the conditioning of the LS problem” [22, p.126].Other desiderata pertain.(i)
“Because rewriting the model (in standardized variables) does not affect any of the implicit estimates, it has no effect on the amount of information contained in the data” [23, p.76].(ii)
Consequences of ill-advised diagnostics can be severe. Degraded numerical accuracy traces to near collinearity of regressors with the constant vector. In short, centering fails to prevent a loss in numerical accuracy; centered diagnostics are unable to discern these potential accuracy problems, whereas uncentered diagnostics are seen to work well. Two widely used statistical packages, SAS and SPSS-X, fail to detect this type of ill-conditioning through use of centered diagnostics and thus return highly inaccurate coefficient estimates. For further details see [3].On balance, for models inM0 the jury is out regarding the use of centered or uncentered diagnostics, to include VIFs. Even Belsley [1] (and elsewhere) concedes circumstances where centering does achieve structurally interpretable models. Of note is that the foregoing listed citations to Belsley apply strictly to condition numbers c2(X); other purveyors of ill-conditioning, specifically VIFs, are not treated explicitly.


### 3.4. A Synthesis

It bears notice that (i) the origin, however remote from the cluster of regressors, is essential for prediction, and (ii) the prediction variance is invariant to parametrizing in centered or uncentered forms. Additional remarks are codified next for subsequent referral.Remark 2.
TypicallyY represents response to input variables {X1,X2,…,Xk}. In a controlled experiment, levels are determined beforehand by subject-matter considerations extraneous to the experiment, to include minimal values. However remote the origin on the basic data scales,  it seems informative in such circumstances to identify the origin with these minima. In such cases the intercept is often of singular interest, since β0 is then the standard against which changes in Y are to be gaged as regressors vary. We adopt this convention in subsequent studies from the archival literature.Remark 3.
In summary, the divergence in views, whether to center or not, appears to be that critical aspects of ill-conditioning, known and widely accepted for models inMw, have been expropriated over decades, mindlessly and without verification, to apply point-by-point for models in M0.


## 3.1. Background

Aspects of ill-conditioning span a considerable literature over many decades. Regarding{Y=Xβ+ϵ}, scaling columns of X to equal lengths approximately minimizes the condition number c2(X) [12, p.120] based on [13]. Nonetheless, c2(X) is cast in [9] as a blunt instrument for ill-conditioning, prompting the need for VIFs and other local constructs. Stewart [9] credits VIFs in concept to Daniel and in name to Marquardt.Ill-conditioning points beyondOLS in view of difficulties cited earlier. Remedies proffered in [14, 15] include transforming variables, adding new data, and deleting variable(s) after checking critical features of the reduced model. Other intended palliatives include Ridge and Partial Least Squares, as compared in [16]; Principal Components regression; Surrogate models as in [17]. All are intended to return reduced standard errors at the expense of bias. Moreover, Surrogate solutions more closely resemble those from an orthogonal system than Ridge [18]. Together the foregoing and other options comprise a substantial literature as collateral to, but apart from, the present study.


## 3.2. To Center

Advocacy for centering includes the following.(i)
VIFs often are defined categorically as the diagonals of the inverse of the correlation matrix of scaled and centered regressors; see [4, 9, 11, 19] for example. These are VIFcs, widely adopted without justification as default to the exclusion of VIFus.(ii)
It is asserted [4] that “centering removes the nonessential ill-conditioning, thus reducing the variance inflation in the coefficient estimates.”(iii)
Centering is advocated when predictor variables are far removed from origins on the basic data scales [10, 11].


## 3.3. Not to Center

Advocacy for uncentered diagnostics includes the following caveats from proponents of centering.(i)
Uncentered data should be examined only if an estimate of the intercept is of interest [9, 10, 20].(ii)
“If the domain of prediction includes the full range from the natural origin through the range of data, the collinearity diagnostics should not be mean centered” [10, p.84].Other issues against centering derive in part from numerical analysis and work by Belsley.(i)
Belsley [1] identifies c2(X) for a system {Y=Xβ+ϵ} as “the potential relative change in the LS solution b that can result from a small relative change in the data.”(ii)
These requirestructurally interpretable variables as “ones whose numerical values and (relative) differences derive meaning and interpretability from knowledge of the structure of the underlying ‘reality’ being modeled” [1, p.75].(iii)
“There is no such thing as ‘nonessential’ ill-conditioning,” and “mean-centering can remove from the data the information needed to assess conditioning correctly” [1, p.74].(iv)
“Collinearity with the intercept can quite generally corrupt the estimates ofall parameters in the model whether or not the intercept is itself of interest and whether or not the data have been centered” [21, p.90].(v)
An example [22, p.121] gives severely ill-conditioned data perfectly conditioned in centered form: “centering alters neither” inflated variances nor extremely sensitive parameter estimates in the basic data; moreover, “diagnosing the conditioning of the centered data (which are perfectly conditioned) would completely overlook this situation, whereas diagnostics based on the basic data would not.”(vi)
To continue from [22], ill-conditioning persists in the propagation of disturbances, in that “a 1 percent relative change in the Xi’s results in over a 40% relative change in the estimates,” despite perfect conditioning in centered form, and “knowledge of the effect of small relative changes in the centered data is not meaningful for assessing the sensitivity of the basic LS problem,” since relative changes and their meanings in centered and uncentered data often differ markedly.(vii)
Regarding choice of origin, “(1) the investigator must be able to pick an origin against which small relative changes can be appropriately assessed and (2) it is the data measured relative to this origin that are relevant to diagnosing the conditioning of the LS problem” [22, p.126].Other desiderata pertain.(i)
“Because rewriting the model (in standardized variables) does not affect any of the implicit estimates, it has no effect on the amount of information contained in the data” [23, p.76].(ii)
Consequences of ill-advised diagnostics can be severe. Degraded numerical accuracy traces to near collinearity of regressors with the constant vector. In short, centering fails to prevent a loss in numerical accuracy; centered diagnostics are unable to discern these potential accuracy problems, whereas uncentered diagnostics are seen to work well. Two widely used statistical packages, SAS and SPSS-X, fail to detect this type of ill-conditioning through use of centered diagnostics and thus return highly inaccurate coefficient estimates. For further details see [3].On balance, for models inM0 the jury is out regarding the use of centered or uncentered diagnostics, to include VIFs. Even Belsley [1] (and elsewhere) concedes circumstances where centering does achieve structurally interpretable models. Of note is that the foregoing listed citations to Belsley apply strictly to condition numbers c2(X); other purveyors of ill-conditioning, specifically VIFs, are not treated explicitly.


## 3.4. A Synthesis

It bears notice that (i) the origin, however remote from the cluster of regressors, is essential for prediction, and (ii) the prediction variance is invariant to parametrizing in centered or uncentered forms. Additional remarks are codified next for subsequent referral.Remark 2.
TypicallyY represents response to input variables {X1,X2,…,Xk}. In a controlled experiment, levels are determined beforehand by subject-matter considerations extraneous to the experiment, to include minimal values. However remote the origin on the basic data scales,  it seems informative in such circumstances to identify the origin with these minima. In such cases the intercept is often of singular interest, since β0 is then the standard against which changes in Y are to be gaged as regressors vary. We adopt this convention in subsequent studies from the archival literature.Remark 3.
In summary, the divergence in views, whether to center or not, appears to be that critical aspects of ill-conditioning, known and widely accepted for models inMw, have been expropriated over decades, mindlessly and without verification, to apply point-by-point for models in M0.


## 4. The Structure of Orthogonality

This section develops the foundations for Reference models capturing orthogonalities of typesV⊥ and C⊥. Essential collateral results are given in support as well.


### 4.1. Collinearity Indices

Stewart [9] reexamines ill-conditioning from the perspective of numerical analysis. Details follow, where X is a generic matrix of regressors having columns {xj;1≤j≤p} and X†=(X′X)-1X′ is the generalized inverse of note, having {xj†;1≤j≤p} as its typical rows. Each corresponding collinearity indexκ is defined in [9, p.72] as
(7){κj=∥xj∥·∥xj†∥;1≤j≤p},
constructed so as to be scale invariant. Observe that ∥xj†∥2 is found along the principal diagonal of [(X†)(X†)′]=(X′X)-1. When X(n×k) in X0=[1n,X] is centered and scaled, Section 3 of [9] shows that the centered collinearity indices κc satisfy {κcj2=VIFc(β^j);1≤j≤k}. In M0 with parameters (β0,β), values corresponding to ∥xj†∥2 from X0†=(X0′X0)-1X0′ lie along the principal diagonal of (X0′X0)-1; the uncentered collinearity indices κu now satisfy {κuj2=∥xj∥2·∥xj†∥2;j=0,1,…,k}. In particular, since x0=1n, we have κu02=n∥x0†∥2. Moreover, in M0 it follows that the uncentered VIFus are squares of the collinearity indices, that is, {VIFu(β^j)=κuj2;j=0,1,…,k}. Note the asymmetry that VIFcs exclude the intercept, in contrast to the inclusive VIFus. That the label Variance Inflation Factors for the latter is a misnomer is covered in Remark 1. Nonetheless, we continue the familiar notation {VIFu(β^j)=κuj2;j=0,1,…,k}.Transcending the essential developments of [9] are connections between collinearity indices and angles between subspaces. To these ends choose a typical xj in X0, and rearrange X0 as [xj,X[j]]. We next seek elements of
(8)Qj′(X0′X0)-1Qj=[xj′xjxj′X[j]X[j]′xjX[j]′X[j]]-1
as reordered by the permutation matrix Qj. From the clockwise rule the (1,1) element of the inverse is
(9)[xj′(In-Pj)xj]-1=[xj′xj-xj′Pjxj]-1=∥xj†∥2
in succession for each {j=0,1,…,k}, where Pj=X[j][X[j]′X[j]]-1X[j]′ is the projection operator onto the subspace 𝒮p(X[j])⊂ℝn spanned by the columns of  X[j]. These in turn enable us to connect {κuj2;j=0,1…,k}, and similarly for centered values {κcj2;j=1…,k}, to the geometry of ill-conditioning as follows.Theorem 4.
For models inM0 let {VIFu(β^j)=κuj2;j=0,1,…,k} be conventional uncentered VIFus in terms of Stewart’s [9] uncentered collinearity indices. These in turn quantify the extent of collinearities between subspaces through angles (in deg) as follows.(i)
Angles between(xj,𝒮p(X[j])) are given by θj=arccos[(1-1/κuj2)1/2], in succession for {j=0,1,…,k}.(ii)
In particular,θ0=arccos[(1-1/κu02)1/2] quantifies the degree of collinearity between the regressors and the constant vector.(iii)
Similarly letBnX=Z=[z1,…,zk] be regressors centered to their means, rearrange as [zj,Z[j]], and let {VIFc(β^j)=κcj2;j=1,…,k} be centered VIFcs in terms of Stewart’s centered collinearity indices. Then angles (in deg) between (zj,𝒮p(Z[j])) are given by {θj=arccos[(1-1/κcj2)1/2];1≤j≤k}.Proof.
From the geometry of the right triangle formed by(xj,Pjxj), the squared lengths satisfy ∥xj∥2=∥Pjxj∥2+∥xj-Pjxj∥2, where ∥xj-Pjxj∥2=RSj is the residual sum of squares from the projection. It follows that the principal angle between (xj,Pjxj) is given by
(10)cos(θj)=xj′Pjxj∥xj∥·∥Pjxj∥=∥Pjxj∥∥xj∥=1-RSj∥xj∥2=1-1κuj2
for {j=0,1,…,k}, to give conclusion (i). Conclusion (ii) follows on specializing (x0,P0x0) with x0=1n and P0=X(X′X)-1X′. Conclusion (iii) follows similarly from the geometry of the right triangle formed by (zj,Pjzj), where Pj now is the projection operator onto the subspace 𝒮p(Z[j])⊂ℝn spanned by the columns of Z[j], to complete our proof.Remark 5.
Rules of thumb in common use for problematicVIFs include those exceeding 10 or even 4; see [11, 24] for example. In angular measure these correspond, respectively, to θj<18.435deg and θj<30.0deg.


### 4.2. Reference Models

We seek as Reference feasible models encoding orthogonalities of typesV⊥ and C⊥. The keys are as follows: (i) to retain essentials of the experimental structure and (ii) to alter what may be changed to achieve orthogonality. For a model in Mw with moment matrix X′X, our opening paragraph prescribes as reference the model RV=D=Diag(d11,…,dpp), as diagonal elements of  X′X, for assessing V⊥-orthogonality. Moreover, on scaling columns of X to equal lengths, RV is perfectly conditioned with c1(RV)=1.0. In addition, every model in Mw clearly conforms with its Reference, in the sense that RV is positive definite, as distinct from models in M0 to follow.Consider again models inM0 as in (6); let C=(X′X-M) with M=nX-X-′ as the mean-centering matrix; and again let D=Diag(d11,…,dkk) comprise the diagonal elements of X′X.(i)
V
⊥
Reference Model. The uncentered VIFus in M0, defined as ratios of diagonal elements of (X0′X0)-1 to reciprocals of diagonal elements of  X0′X0, appear to have seen exclusive usage, apparently in keeping with Remark 3. However, the following disclaimer must be registered as the formal equivalent of Remark 1.Theorem 6.
Statements that conventionalVIFus quantify variance inflation owing to nondiagonal X0′X0 are false for models in M0 having X-≠0.Proof.
Since the Reference variances are reciprocals of diagonal elements ofX0′X0, this usage is predicated on the false postulate that X0′X0 can be diagonal for X-≠0. Specifically, [1n,X] are linearly independent, that is, 1n′X=0, if and only if  X has been mean centered beforehand.To the contrary, Gunst [25] purports to show that VIFu(β^0) registers genuine variance inflation, namely, the price to be paid in variance for designing an experiment having X-≠0, as opposed to X-=0. Since variances for intercepts are Var(·∣X-=0)=σ2/n and Var(·∣X-≠0)=σ2c00≥σ2/n from (6), their ratio is shown in [25] to be κu02=nc00≥1 in the parlance of Section 2.3. We concede this to be a ratio of variances but, to the contrary, not a VIF, since the parameters differ. In particular, Var(β^0∣X-≠0)=σ2c00, whereas Var(α^∣X-≠0)=σ2/n, with α=β0+β1X-1+⋯+βkX-k in centered regressors. Nonetheless, we still find the conventional VIFu(β^0) to be useful for purposes to follow.Remark 7.
Section3 highlights continuing concern in regard to collinearity of regressors with the constant vector. Theorem 4(ii) and expression (10) support the use of θ0=arccos[(1-1/κu02)1/2] as an informative gage on the extent of this occurrence. Specifically, the smaller the angle, the greater the extent of such collinearity.
Instead of conventionalVIFus given the foregoing disclaimer, we have the following amended version as Reference for uncentered diagnostics, altering what may be changed but leaving X-≠0 intact.Definition 8.
Given a model inM0 with second-moment matrix X0′X0, the amended Reference model for assessing V⊥-orthogonality is RV=[nnX-′nX-D] with D=Diag(d11,…,dkk) as diagonal elements of X′X, provided that RV is positive definite. We identify a model to be V⊥-orthogonal when X0′X0=RV.
As anticipated, a prospectiveRV fails to conform to experimental data if not positive definite. These and further prospects are covered in the following, where ϕi designates the angle between {(1n,Xi);i=1,…,k}.Lemma 9.
TakeRV as a prospective Reference for V⊥-orthogonality.(i)
In order thatRV maybe positive definite, it is necessary that {nX-′D-1X-<1}, that is, that {n∑i=1k‍(X-i2/dii)<1}.(ii)
Equivalently, it is necessary that∑i=1kcos2(ϕi)<1, with ϕi as the angle between {(1n,Xi);i=1,…,k}.(iii)
The Reference variance forβ^0 is Var(β^0∣X′X=D)=[n(1-nX-′D-1X-)]-1.(iv)
The Reference variances for slopes are given by(11){Var(β^i∣X′X=D)=1dii[1+γX-i2dii];1≤i≤k},where  γ=n/(1-nX-′D-1X-).Proof.
The clockwise rule for determinants gives|RV|=|D|[n-n2X-′D-1X-]. Conclusion (i) follows since |D|>0. The computation
(12)cos(ϕi)=1n′Xi∥1n∥∥Xi∥=1n′Xindii=nX-i2dii,
in parallel with (10), gives conclusion (ii). Using the clockwise rule for block-partitioned inverses, the (1,1) element of RV-1 is given by conclusion (iii). Similarly, the lower right block of RV-1, of order (k×k), is the inverse of H=D-nX-X-′. On identifying a=b=X-,  α=n, and {ai*=ai/dii} in Theorem 8.3.3 of [26], we have that H-1=D-1+γD-1X-X-′D-1. Conclusion (iv) follows on extracting its diagonal elements, to complete our proof.Corollary 10.
For the casek=2, in order that RV maybe positive definite, it is necessary that ϕ1+ϕ2>90deg.Proof.
Beginning with Lemma9(ii), compute
(13)cos(ϕ1+ϕ2)=cosϕ1cosϕ2-sinϕ1sinϕ2=cosϕ1cosϕ2-[1-(cos2ϕ1+cos2ϕ2)+cos2ϕ1cos2ϕ2,]1/2
which is <0 when 1-(cos2ϕ1+cos2ϕ2)>0.Moreover, the matrixRV itself is intrinsically ill conditioned owing to X-≠0, its condition number depending on X-. To quantify this dependence, we have the following, where columns of X have been standardized to common lengths {|Xi|=c;1≤i≤k}.Lemma 11.
LetRV=[nT′TcIk] as in Definition 8, with T=nX-, D=cIk, and c>0.(i)
The ordered eigenvalues ofRV are [λ1,c,…,c,λp], where p=k+1 and (λ1,λp) are the roots of  λ∈[(n+c)±(n-c)2+4τ2]/2 and τ2=T′T.(ii)
The roots are positive, andRV is positive definite, if and only if (nc-τ2)>0.(iii)
IfRV is positive definite, its condition number is c1(RV)=λ1/λp and is increasing in τ2.Proof.
Eigenvalues are roots of the determinantal equation(14)Det[(n-λ)T′T(c-λ)Ik]=0⟺(c-λ)k-1[(n-λ)(c-λ)-T′T]=0
from the clockwise rule, giving (k-1) values {λ=c} and two roots of the quadratic equation {λ2-(n+c)λ+nc-τ2=0}, to give conclusion (i). Conclusion (ii) holds since the product of roots of the quadratic equation is (nc-τ2) and the greater root is positive. Conclusion (iii) follows directly, to complete our proof.(ii)
C
⊥
Reference Model. As noted, the notion of C⊥-orthogonality applies exclusively for models in M0.  Accordingly, as Reference we seek to alter X′X so that the matrix comprising sums of squares and sums of products of deviations from means, thus altered, is diagonal. To achieve this canon of C⊥-orthogonality, and to anticipate notation for subsequent use, we have the following.Definition 12.
(i) For a model inM0 with second-moment matrix X0′X0 and inverse as in (6), let Cw=(W-M), and identify an indicator vector 𝔾 = [g12,g13,…,g1k,g23,…,  g2k,…,gk-1,k] of order ((k-1)×(k-2)) in lexicographic order, where {gij=0;i≠j} if the (i,j) element of Cw is zero and {gij=1;i≠j} if the (i,j) element of Cw is (Xi′Xj-nX-iX-j), that is, unchanged from C=(X′X-nX-X-′)=Z′Z, with Z=BnX as the regressors centered to their means.
(ii) In particular, the Reference model inM0 for assessing C⊥-orthogonality is RC=[nnX-′nX-W] such that Cw=(W-M) and its inverse G from (6) are diagonal, that is, taking W=[wij] such that {wii=dii;1≤i≤k} and {wij=mij;foralli≠j} or equivalently, 𝔾=[0,0,…,0]. In this case, we identify the model to be C⊥-orthogonal.Recall that conventionalVIFcs for [β^1,β^2,…,β^k] are ratios of diagonal elements of (Z′Z)-1 to reciprocals of the diagonal elements of the centered Z′Z. Apparently this rests on the logic that C⊥-orthogonality in M0 implies that Z′Z is diagonal. However, the converse fails and instead is embodied in RC∈M0. In consequence, conventional VIFcs are deficient in applying only to slopes, whereas the VFcs resulting from Definition 12(ii) apply informatively for all of [β^0,β^1,β^2,…,β^k].Definition 13.
DesignateVFv(·) and VFc(·) as Variance Factors resulting from Reference models of Definition 8 and Definition 12(ii), respectively. On occasion these represent Variance Deflation in addition to VIFs.Essential invariance properties pertain. ConventionalVIFcs are scale invariant; see Section 3 of [9]. We see next that they are translation invariant as well. To these ends we shift the columns of X=[X1,X2,…,Xk] to a new origin through {X→X-L=H}, where L=[c11n,c21n,…,ck1n]. The resulting model is
(15){Y=(β01n+Lβ)+Hβ+ϵ}⟺{Y=(β0+ψ)1n+Hβ+ϵ},
thus preserving slopes, where ψ=c′β=(c1β1+c2β2+⋯+ckβk). Corresponding to X0=[1n,X] is U0=[1n,H] and to X0′X0 and its inverse are
(16)U0′U0=[n1n′HH′1nH′H],(U0′U0)-1=[b00b01b01′G]
in the form of (6). This pertains to subsequent developments, and basic invariance results emerge as follows.Lemma 14.
Consider{Y=β01n+Xβ+ϵ} together with the shifted version {Y=(β0+ψ)1n+Hβ+ε}, both in M0. Then the matrices G appearing in (6) and (16) are identical.Proof.
Rules for block-partitioned inverses again assert thatG of (16) is the inverse of
(17)(H′H-n-1H′1n1n′H)=H′(In-n-11n1n′)H=H′BnH=X′BnX
since BnL=0, to complete our proof.These facts in turn support subsequent claims that centeredVIFcs are translation and scale invariant for slopes β^′=[β^1,β^2,…,β^k], apart from the intercept β^0.


### 4.3. A Critique

Again we distinguishVIFcs and VIFus from centered and uncentered regressors. The following comments apply.(C1)
A design is eitherV⊥ or C⊥-orthogonal, respectively, according as the lower right (k×k) block X′X of X0′X0, or C=(X′X-nX-X-′) from expression (6), is diagonal. Orthogonalities of type V⊥ and C⊥ are exclusive and hence work at crossed purposes.(C2)
In particular,C⊥-orthogonality holds if and only if the columns of  X are V⊥-nonorthogonal. If  X is C⊥-orthogonal, then for i≠j,  Cw(ij)=Xi′Xj-nX-iX-j=0 and Xi′Xj≠0. Conversely, if X indeed is V⊥-orthogonal, so that X′X =  D=  Diag(d11,…,dkk), then (D-nX-X-′) cannot be diagonal as Reference, in which case the conventional VIFcs are tenuous.(C3)
ConventionalVIFus, based on uncentered X in M0 with X-≠0, do not gage variance inflation as claimed, founded on the false tenet that X0′X0 can be diagonal.(C4)
To detect influential observations and to classify high leverage points, case–influence diagnostics, namely{fjI=VIFj(-I)/VIFj;1≤j≤k}, are studied in [27] for assessing the impact of subsets on variance inflation. Here VIFj is from the full data and VIFj(-I) on deleting observations in the index set I. Similarly, [28] proposes using VIF(i) on deleting the ith observation. In the present context these would gain substance on modifying VFv and VFc accordingly.


## 4.1. Collinearity Indices

Stewart [9] reexamines ill-conditioning from the perspective of numerical analysis. Details follow, where X is a generic matrix of regressors having columns {xj;1≤j≤p} and X†=(X′X)-1X′ is the generalized inverse of note, having {xj†;1≤j≤p} as its typical rows. Each corresponding collinearity indexκ is defined in [9, p.72] as
(7){κj=∥xj∥·∥xj†∥;1≤j≤p},
constructed so as to be scale invariant. Observe that ∥xj†∥2 is found along the principal diagonal of [(X†)(X†)′]=(X′X)-1. When X(n×k) in X0=[1n,X] is centered and scaled, Section 3 of [9] shows that the centered collinearity indices κc satisfy {κcj2=VIFc(β^j);1≤j≤k}. In M0 with parameters (β0,β), values corresponding to ∥xj†∥2 from X0†=(X0′X0)-1X0′ lie along the principal diagonal of (X0′X0)-1; the uncentered collinearity indices κu now satisfy {κuj2=∥xj∥2·∥xj†∥2;j=0,1,…,k}. In particular, since x0=1n, we have κu02=n∥x0†∥2. Moreover, in M0 it follows that the uncentered VIFus are squares of the collinearity indices, that is, {VIFu(β^j)=κuj2;j=0,1,…,k}. Note the asymmetry that VIFcs exclude the intercept, in contrast to the inclusive VIFus. That the label Variance Inflation Factors for the latter is a misnomer is covered in Remark 1. Nonetheless, we continue the familiar notation {VIFu(β^j)=κuj2;j=0,1,…,k}.Transcending the essential developments of [9] are connections between collinearity indices and angles between subspaces. To these ends choose a typical xj in X0, and rearrange X0 as [xj,X[j]]. We next seek elements of
(8)Qj′(X0′X0)-1Qj=[xj′xjxj′X[j]X[j]′xjX[j]′X[j]]-1
as reordered by the permutation matrix Qj. From the clockwise rule the (1,1) element of the inverse is
(9)[xj′(In-Pj)xj]-1=[xj′xj-xj′Pjxj]-1=∥xj†∥2
in succession for each {j=0,1,…,k}, where Pj=X[j][X[j]′X[j]]-1X[j]′ is the projection operator onto the subspace 𝒮p(X[j])⊂ℝn spanned by the columns of  X[j]. These in turn enable us to connect {κuj2;j=0,1…,k}, and similarly for centered values {κcj2;j=1…,k}, to the geometry of ill-conditioning as follows.Theorem 4.
For models inM0 let {VIFu(β^j)=κuj2;j=0,1,…,k} be conventional uncentered VIFus in terms of Stewart’s [9] uncentered collinearity indices. These in turn quantify the extent of collinearities between subspaces through angles (in deg) as follows.(i)
Angles between(xj,𝒮p(X[j])) are given by θj=arccos[(1-1/κuj2)1/2], in succession for {j=0,1,…,k}.(ii)
In particular,θ0=arccos[(1-1/κu02)1/2] quantifies the degree of collinearity between the regressors and the constant vector.(iii)
Similarly letBnX=Z=[z1,…,zk] be regressors centered to their means, rearrange as [zj,Z[j]], and let {VIFc(β^j)=κcj2;j=1,…,k} be centered VIFcs in terms of Stewart’s centered collinearity indices. Then angles (in deg) between (zj,𝒮p(Z[j])) are given by {θj=arccos[(1-1/κcj2)1/2];1≤j≤k}.Proof.
From the geometry of the right triangle formed by(xj,Pjxj), the squared lengths satisfy ∥xj∥2=∥Pjxj∥2+∥xj-Pjxj∥2, where ∥xj-Pjxj∥2=RSj is the residual sum of squares from the projection. It follows that the principal angle between (xj,Pjxj) is given by
(10)cos(θj)=xj′Pjxj∥xj∥·∥Pjxj∥=∥Pjxj∥∥xj∥=1-RSj∥xj∥2=1-1κuj2
for {j=0,1,…,k}, to give conclusion (i). Conclusion (ii) follows on specializing (x0,P0x0) with x0=1n and P0=X(X′X)-1X′. Conclusion (iii) follows similarly from the geometry of the right triangle formed by (zj,Pjzj), where Pj now is the projection operator onto the subspace 𝒮p(Z[j])⊂ℝn spanned by the columns of Z[j], to complete our proof.Remark 5.
Rules of thumb in common use for problematicVIFs include those exceeding 10 or even 4; see [11, 24] for example. In angular measure these correspond, respectively, to θj<18.435deg and θj<30.0deg.


## 4.2. Reference Models

We seek as Reference feasible models encoding orthogonalities of typesV⊥ and C⊥. The keys are as follows: (i) to retain essentials of the experimental structure and (ii) to alter what may be changed to achieve orthogonality. For a model in Mw with moment matrix X′X, our opening paragraph prescribes as reference the model RV=D=Diag(d11,…,dpp), as diagonal elements of  X′X, for assessing V⊥-orthogonality. Moreover, on scaling columns of X to equal lengths, RV is perfectly conditioned with c1(RV)=1.0. In addition, every model in Mw clearly conforms with its Reference, in the sense that RV is positive definite, as distinct from models in M0 to follow.Consider again models inM0 as in (6); let C=(X′X-M) with M=nX-X-′ as the mean-centering matrix; and again let D=Diag(d11,…,dkk) comprise the diagonal elements of X′X.(i)
V
⊥
Reference Model. The uncentered VIFus in M0, defined as ratios of diagonal elements of (X0′X0)-1 to reciprocals of diagonal elements of  X0′X0, appear to have seen exclusive usage, apparently in keeping with Remark 3. However, the following disclaimer must be registered as the formal equivalent of Remark 1.Theorem 6.
Statements that conventionalVIFus quantify variance inflation owing to nondiagonal X0′X0 are false for models in M0 having X-≠0.Proof.
Since the Reference variances are reciprocals of diagonal elements ofX0′X0, this usage is predicated on the false postulate that X0′X0 can be diagonal for X-≠0. Specifically, [1n,X] are linearly independent, that is, 1n′X=0, if and only if  X has been mean centered beforehand.To the contrary, Gunst [25] purports to show that VIFu(β^0) registers genuine variance inflation, namely, the price to be paid in variance for designing an experiment having X-≠0, as opposed to X-=0. Since variances for intercepts are Var(·∣X-=0)=σ2/n and Var(·∣X-≠0)=σ2c00≥σ2/n from (6), their ratio is shown in [25] to be κu02=nc00≥1 in the parlance of Section 2.3. We concede this to be a ratio of variances but, to the contrary, not a VIF, since the parameters differ. In particular, Var(β^0∣X-≠0)=σ2c00, whereas Var(α^∣X-≠0)=σ2/n, with α=β0+β1X-1+⋯+βkX-k in centered regressors. Nonetheless, we still find the conventional VIFu(β^0) to be useful for purposes to follow.Remark 7.
Section3 highlights continuing concern in regard to collinearity of regressors with the constant vector. Theorem 4(ii) and expression (10) support the use of θ0=arccos[(1-1/κu02)1/2] as an informative gage on the extent of this occurrence. Specifically, the smaller the angle, the greater the extent of such collinearity.
Instead of conventionalVIFus given the foregoing disclaimer, we have the following amended version as Reference for uncentered diagnostics, altering what may be changed but leaving X-≠0 intact.Definition 8.
Given a model inM0 with second-moment matrix X0′X0, the amended Reference model for assessing V⊥-orthogonality is RV=[nnX-′nX-D] with D=Diag(d11,…,dkk) as diagonal elements of X′X, provided that RV is positive definite. We identify a model to be V⊥-orthogonal when X0′X0=RV.
As anticipated, a prospectiveRV fails to conform to experimental data if not positive definite. These and further prospects are covered in the following, where ϕi designates the angle between {(1n,Xi);i=1,…,k}.Lemma 9.
TakeRV as a prospective Reference for V⊥-orthogonality.(i)
In order thatRV maybe positive definite, it is necessary that {nX-′D-1X-<1}, that is, that {n∑i=1k‍(X-i2/dii)<1}.(ii)
Equivalently, it is necessary that∑i=1kcos2(ϕi)<1, with ϕi as the angle between {(1n,Xi);i=1,…,k}.(iii)
The Reference variance forβ^0 is Var(β^0∣X′X=D)=[n(1-nX-′D-1X-)]-1.(iv)
The Reference variances for slopes are given by(11){Var(β^i∣X′X=D)=1dii[1+γX-i2dii];1≤i≤k},where  γ=n/(1-nX-′D-1X-).Proof.
The clockwise rule for determinants gives|RV|=|D|[n-n2X-′D-1X-]. Conclusion (i) follows since |D|>0. The computation
(12)cos(ϕi)=1n′Xi∥1n∥∥Xi∥=1n′Xindii=nX-i2dii,
in parallel with (10), gives conclusion (ii). Using the clockwise rule for block-partitioned inverses, the (1,1) element of RV-1 is given by conclusion (iii). Similarly, the lower right block of RV-1, of order (k×k), is the inverse of H=D-nX-X-′. On identifying a=b=X-,  α=n, and {ai*=ai/dii} in Theorem 8.3.3 of [26], we have that H-1=D-1+γD-1X-X-′D-1. Conclusion (iv) follows on extracting its diagonal elements, to complete our proof.Corollary 10.
For the casek=2, in order that RV maybe positive definite, it is necessary that ϕ1+ϕ2>90deg.Proof.
Beginning with Lemma9(ii), compute
(13)cos(ϕ1+ϕ2)=cosϕ1cosϕ2-sinϕ1sinϕ2=cosϕ1cosϕ2-[1-(cos2ϕ1+cos2ϕ2)+cos2ϕ1cos2ϕ2,]1/2
which is <0 when 1-(cos2ϕ1+cos2ϕ2)>0.Moreover, the matrixRV itself is intrinsically ill conditioned owing to X-≠0, its condition number depending on X-. To quantify this dependence, we have the following, where columns of X have been standardized to common lengths {|Xi|=c;1≤i≤k}.Lemma 11.
LetRV=[nT′TcIk] as in Definition 8, with T=nX-, D=cIk, and c>0.(i)
The ordered eigenvalues ofRV are [λ1,c,…,c,λp], where p=k+1 and (λ1,λp) are the roots of  λ∈[(n+c)±(n-c)2+4τ2]/2 and τ2=T′T.(ii)
The roots are positive, andRV is positive definite, if and only if (nc-τ2)>0.(iii)
IfRV is positive definite, its condition number is c1(RV)=λ1/λp and is increasing in τ2.Proof.
Eigenvalues are roots of the determinantal equation(14)Det[(n-λ)T′T(c-λ)Ik]=0⟺(c-λ)k-1[(n-λ)(c-λ)-T′T]=0
from the clockwise rule, giving (k-1) values {λ=c} and two roots of the quadratic equation {λ2-(n+c)λ+nc-τ2=0}, to give conclusion (i). Conclusion (ii) holds since the product of roots of the quadratic equation is (nc-τ2) and the greater root is positive. Conclusion (iii) follows directly, to complete our proof.(ii)
C
⊥
Reference Model. As noted, the notion of C⊥-orthogonality applies exclusively for models in M0.  Accordingly, as Reference we seek to alter X′X so that the matrix comprising sums of squares and sums of products of deviations from means, thus altered, is diagonal. To achieve this canon of C⊥-orthogonality, and to anticipate notation for subsequent use, we have the following.Definition 12.
(i) For a model inM0 with second-moment matrix X0′X0 and inverse as in (6), let Cw=(W-M), and identify an indicator vector 𝔾 = [g12,g13,…,g1k,g23,…,  g2k,…,gk-1,k] of order ((k-1)×(k-2)) in lexicographic order, where {gij=0;i≠j} if the (i,j) element of Cw is zero and {gij=1;i≠j} if the (i,j) element of Cw is (Xi′Xj-nX-iX-j), that is, unchanged from C=(X′X-nX-X-′)=Z′Z, with Z=BnX as the regressors centered to their means.
(ii) In particular, the Reference model inM0 for assessing C⊥-orthogonality is RC=[nnX-′nX-W] such that Cw=(W-M) and its inverse G from (6) are diagonal, that is, taking W=[wij] such that {wii=dii;1≤i≤k} and {wij=mij;foralli≠j} or equivalently, 𝔾=[0,0,…,0]. In this case, we identify the model to be C⊥-orthogonal.Recall that conventionalVIFcs for [β^1,β^2,…,β^k] are ratios of diagonal elements of (Z′Z)-1 to reciprocals of the diagonal elements of the centered Z′Z. Apparently this rests on the logic that C⊥-orthogonality in M0 implies that Z′Z is diagonal. However, the converse fails and instead is embodied in RC∈M0. In consequence, conventional VIFcs are deficient in applying only to slopes, whereas the VFcs resulting from Definition 12(ii) apply informatively for all of [β^0,β^1,β^2,…,β^k].Definition 13.
DesignateVFv(·) and VFc(·) as Variance Factors resulting from Reference models of Definition 8 and Definition 12(ii), respectively. On occasion these represent Variance Deflation in addition to VIFs.Essential invariance properties pertain. ConventionalVIFcs are scale invariant; see Section 3 of [9]. We see next that they are translation invariant as well. To these ends we shift the columns of X=[X1,X2,…,Xk] to a new origin through {X→X-L=H}, where L=[c11n,c21n,…,ck1n]. The resulting model is
(15){Y=(β01n+Lβ)+Hβ+ϵ}⟺{Y=(β0+ψ)1n+Hβ+ϵ},
thus preserving slopes, where ψ=c′β=(c1β1+c2β2+⋯+ckβk). Corresponding to X0=[1n,X] is U0=[1n,H] and to X0′X0 and its inverse are
(16)U0′U0=[n1n′HH′1nH′H],(U0′U0)-1=[b00b01b01′G]
in the form of (6). This pertains to subsequent developments, and basic invariance results emerge as follows.Lemma 14.
Consider{Y=β01n+Xβ+ϵ} together with the shifted version {Y=(β0+ψ)1n+Hβ+ε}, both in M0. Then the matrices G appearing in (6) and (16) are identical.Proof.
Rules for block-partitioned inverses again assert thatG of (16) is the inverse of
(17)(H′H-n-1H′1n1n′H)=H′(In-n-11n1n′)H=H′BnH=X′BnX
since BnL=0, to complete our proof.These facts in turn support subsequent claims that centeredVIFcs are translation and scale invariant for slopes β^′=[β^1,β^2,…,β^k], apart from the intercept β^0.


## 4.3. A Critique

Again we distinguishVIFcs and VIFus from centered and uncentered regressors. The following comments apply.(C1)
A design is eitherV⊥ or C⊥-orthogonal, respectively, according as the lower right (k×k) block X′X of X0′X0, or C=(X′X-nX-X-′) from expression (6), is diagonal. Orthogonalities of type V⊥ and C⊥ are exclusive and hence work at crossed purposes.(C2)
In particular,C⊥-orthogonality holds if and only if the columns of  X are V⊥-nonorthogonal. If  X is C⊥-orthogonal, then for i≠j,  Cw(ij)=Xi′Xj-nX-iX-j=0 and Xi′Xj≠0. Conversely, if X indeed is V⊥-orthogonal, so that X′X =  D=  Diag(d11,…,dkk), then (D-nX-X-′) cannot be diagonal as Reference, in which case the conventional VIFcs are tenuous.(C3)
ConventionalVIFus, based on uncentered X in M0 with X-≠0, do not gage variance inflation as claimed, founded on the false tenet that X0′X0 can be diagonal.(C4)
To detect influential observations and to classify high leverage points, case–influence diagnostics, namely{fjI=VIFj(-I)/VIFj;1≤j≤k}, are studied in [27] for assessing the impact of subsets on variance inflation. Here VIFj is from the full data and VIFj(-I) on deleting observations in the index set I. Similarly, [28] proposes using VIF(i) on deleting the ith observation. In the present context these would gain substance on modifying VFv and VFc accordingly.


## 5. Case Study 1: Continued

### 5.1. The Setting

We continue an elementary and transparent example to illustrate essentials. RecallM0:{Yi=β0+β1X1+β2X2+ϵi} of Section 2.2, the design X0=[15,X1,X2] of order (5×3), and U=X0′X0 and its inverse V=(X0′X0)-1, as in expressions (1). The design is neither V⊥ nor C⊥ orthogonal since neither the lower right (2×2) block of X0′X0 nor the centered Z′Z is diagonal. Moreover, the uncentered VIFus as listed in (3) are not the vaunted relative increases in variances owing to nonorthogonal columns of X0. Indeed, the only opportunity for V⊥-orthogonality here is between columns [X1,X2].Nonetheless, from Section4.1 we utilize Theorem 4(i) and (10) to connect the collinearity indices of [9] to angles between subspaces. Specifically, Minitab recovered values for the residuals {RSj=∥xj-Pjxj∥2;j=0,1,2}; further computations proceed routinely as listed in Table 1. In particular, the principal angle θ0 between the constant vector and the span of the regressor vectors is θ0=31.751deg, as anticipated in Remark 7.Table 1
Values for residualsRSj=∥xj-Pjxj∥2, for  ∥Xj∥2, for κuj2=VIFu(βj^), for (1-1/κuj2)1/2, and for θj, for j=0,1,2.
j
R
S
j
∥
X
j
∥
2
κ
j
2
(
1
-
1
/
κ
u
j
2
)
1
/
2
θ
j (deg)
0
1.3846
5.0
3.6111
0.8503
31.751
1
0.6429
2.5
3.8889
0.8619
30.470
2
2.5714
3.0
1.1667
0.3780
67.792


### 5.2.V⊥-Orthogonality

For subsequent reference letD(x) be a design as in (1) but with second-moment matrix:
(18)U(x)=[53132.5x1x3].
Invoking Definition 8, we seek a V⊥-orthogonal Reference with moment matrix U(0) having X1′X2=0,thatis,D(0). This is found constructively on retaining [15,•,X2] in X0, but replacing X1 by X10=[1,0.5,0.5,1,0]′, as listed in Table 2, giving the design D(0) with columns [15,X10,X2] as Reference.Table 2
Designs{D(x);  x∈[-0.5,0.0,0.5,0.6,1.0,1.5]}, obtained by substituting X10 = [X11,X12,X13,X14,X15]′ in [15, •, X2] of order (5×3), and corresponding variances {Var(β^0),Var(β^1),Var(β^2)}.
Design
X
11
X
12
X
13
X
14
X
15
Var
(
β
^
0
)
Var
(
β
^
1
)
Var
(
β
^
2
)
D
(
-
0.5
)
1.0
0.0
0.5
1.0
0.5
1.9333
3.7333
0.9333
D
(
0.0
)
1.0
0.5
0.5
1.0
0.0
0.9375
1.7500
0.4375
D
(
0.5
)
0.5
1.0
0.0
1.0
0.5
0.7436
1.4359
0.3590
D
(
0.6
)
0.6091
0.5793
0.6298
1.1819
0.000
0.7286
1.4286
0.3570
D
(
1.0
)
0.0
0.5
0.5
1.0
1.0
0.7222
1.5556
0.3889
D
(
1.5
)
0.0
1.0
0.5
0.5
1.0
0.9130
2.4348
0.6087Accordingly,RV=U(0) is “as V⊥-orthogonal as it can get,” given the lengths and sums of (X1,X2) as prescribed in the experiment and as preserved in the Reference model. Lemma 9, with X-=[0.6,0.2]′ and D= Diag(2.5,3), gives X-′D-1X-=0.157333 and γ=5/(1-5X-′D-1X-)=23.4375. Applications of Lemma 9(iii)-(iv) in succession give the reference variances
(19)Var(β^0∣X′X=D)=[5(1-5X-′D-1X-)]-1=0.9375,Var(β^1∣X′X=D)=1d11[1+γX-12d11]=25[1+0.72γ5]=1.7500,Var(β^2∣X′X=D)=1d22[1+γX-22d22]=13[1+0.04γ3]=0.4375.As Reference, these combine with actual variances fromV=[Vij] at (1),  giving  VFvfor  the original design  X0 relative to RV.For example, VFv(β^0) = 0.7222/0.9375 = 0.7704, with   V11=0.7222  from (1), and 0.9375 from (19). This contrasts with VIFu(β^0)=3.6111 in (3) as in Section 2.2 and Table 2, it is noteworthy that the nonorthogonal design D(1) yields uniformly smaller variances than the V⊥-orthogonal RV, namely, D(0).Further versions{D(x);  x∈[-0.5,0.0,0.5,0.6,1.0,1.5]} of our basic design are listed for comparison in Table 2, where x = X1′X2 for the various designs. The designs themselves are exhibited on transposing rows X1′= [X11,X12,X13,X14,X15] from Table 2, and substituting each into the template X00=[15,•,X2]. The design D(2) was constructed but not listed since its X0′X0 is not invertible. Clearly these are actual designs amenable to experimental deployment.


### 5.3.C⊥-Orthogonality

ContinuingX0=[15,X1,X2] and invoking Definition 12(ii), we seek a C⊥-orthogonal Reference having the matrix Cw as diagonal. This is identified as D(0.6) in Table 2. From this the matrix RC and its inverse are
(20)RC=[53132.50.610.63],RC-1=[0.7286-0.8572-0.0714-0.85711.42860.0-0.07140.00.3571].
The variance factors are listed in (4) where, for example, VFc(β^2) = 0.3889/0.3571 = 1.0889. As distinct from conventional VIFcs for β^1 and β^2 only, our VFc(β^0) here reflects Variance Deflation, wherein β0 is estimated more precisely in the initial C⊥-nonorthogonal design.Remark 15.
Observe that the choice forRC may be posed as seeking RC=U(x) such that RC-1(2,3)=0.0, then solving numerically for x using Maple for example. However, the algorithm in Definition 12(ii) affords a direct solution: that Cw should be diagonal stipulates its off-diagonal element as X1′X2-3/5 = 0, so that X1′X2 = 0.6 in RC at expression (20).To illustrate Theorem4(iii), we compute cos(θ) = (1-1/1.0889)1/2 = 0.2857 and θ = 73.398 deg as the angle between the vectors [X1,X2] of (1) when centered to their means. For slopes in M0, [9] shows that {VIFc(β^j)<VIFu(β^j);1≤i≤k}. This follows since their numerators are equal, but denominators are reciprocals of lengths of the centered and uncentered regressor vectors. To illustrate, numerators are equal for VIFu(β^3)=0.3889/(1/3)=1.1667 and for VIFc(β^3)=0.3889/(1/2.8)=1.0889, but denominators are reciprocals of X2′X2=3 and ∑j=15‍(X2j-X-2)2=2.8.A further note on orthogonality is germane. Suppose that the actual experiment isD(0) yet, towards a thorough diagnosis, the user evaluates the conventional VIFc(β^1)=1.2250=VIFc(β^2) as ratios of variances of D(0) to D(0.6) from Table 2. Unfortunately, their meaning is obscured since a design cannot at once be V⊥ and C⊥-orthogonal.As in Remark2, we next alter the basic design X0 at (1) on shifting columns of the measurements to [0,0] as minima and scaling to have squared lengths equal to n=5. The resulting X0 follows directly from (1). The new matrix U=X0'X0 and its inverse V are
(21)U=[54.24264.24264.2426544.242645],V=[1-0.4714-0.4714-0.47140.7778-0.2222-0.4714-0.22220.7778]
giving conventional VIFus as [5,3.8889,3.8889]. Against C⊥-orthogonality, this gives the diagnostics
(22)[VFc(β^0),VFc(β^1),VFc(β^2)]=[0.8140,1.0889,1.0889]
demonstrating, in comparison with (4), that VFcs, apart from β^0, are invariant under translation and scaling, a consequence of Lemma 14.We further seek to compare variances for the shifted and scaled(X1,X2) against V⊥-orthogonality as Reference. However, from U=X0′X0 at (21) we determine that X-1=0.84853=X-2 and 5[(X-12/5)+(X-22/5)]=1.4400. Since the latter exceeds unity, we infer from Lemma 9(i) that V⊥-orthogonality is incompatible with this configuration of the data, so that the VFvs are undefined. Equivalently, on evaluating ϕ1=ϕ2=22.90deg and ϕ1+ϕ2=45.80<90deg, Corollary 10 asserts that RV is not positive definite. This appears to be anomalous, until we recall that V⊥-orthogonality is invariant under rescaling, but not recentering the regressors.


### 5.4. A Critique

We reiterate apparent ambiguity ascribed in the literature toorthogonality. A number of widely held guiding precepts has evolved, as enumerated in part in our opening paragraph and in Section 3. As in Remark 3, these clearly have been taken to apply verbatim for X0 and X0′X0 in M0, to be paraphrased in part as follows.(P1)
Ill-conditioning espouses inflated variances; that is,VIFs necessarily equal or exceed unity.(P2)
C⊥-orthogonal designs are “ideal” in that VIFcs for such designs are all unity; see [11] for example.We next reassess these precepts as they play out underV⊥ and C⊥ orthogonality in connection with Table 2.(C1)
For models inM0 having X-≠0, we reiterate that the uncentered VIFus at expression (3), namely [3.6111,3.8889,1.1667], overstate adverse effects on variances of the V⊥-nonorthogonal array, since X0′X0 cannot be diagonal. Moreover, these conventional values fail to discern that the revised values for VFvs at (5), namely, [0.7704,0.8889,0.8889], reflect that [β0,β1,β2] are estimated with greater efficiency in the V⊥-nonorthogonal array D(1).(C2)
ForV⊥-orthogonality, the claim P1 thus is false by counterexample. The VFs at (5) are Variance Deflation Factors for design D(1), where X1′X2=1, relative to the V⊥-orthogonal D(0).(C3)
Additionally, the variances{Var[β^0(x)], Var[β^1(x)], Var[β^2(x)]} in Table 2 are all seen to decrease from D(0):[0.9375, 1.7500, 0.4375] → D(1):[0.7222, 1.5556, 0.3889] despite the transition from the V⊥-orthogonal D(0) to the V⊥-nonorthogonal D(1). Similar trends are confirmed in other ill-conditioned data sets from the literature.(C4)
ForC⊥-orthogonal designs, the claim P2 is false by counterexample. Such designs need not be “ideal,” as β0 may be estimated more efficiently in a C⊥-nonorthogonal design, as demonstrated by the VFcs for β^0 at (4) and (22). Similar trends are confirmed elsewhere, as seen subsequently.(C5)
VFs for β^0 are critical in prediction, where prediction variances necessarily depend on Var(β^0), especially for predicting near the origin of the system of coordinates.(C6)
Dissonance betweenV⊥ and C⊥ is seen in Table 2, where D(0.6), as C⊥-orthogonal with X1′X2=0.6, is the antithesis of V⊥-orthogonality at D(0), where X1′X2=0.(C7)
In short, these transparent examples serve to dispel the decades-old mantra that ill-conditioning necessarily spawns inflated variances for models inM0, and they serve to illuminate the contributing structures.


## 5.1. The Setting

We continue an elementary and transparent example to illustrate essentials. RecallM0:{Yi=β0+β1X1+β2X2+ϵi} of Section 2.2, the design X0=[15,X1,X2] of order (5×3), and U=X0′X0 and its inverse V=(X0′X0)-1, as in expressions (1). The design is neither V⊥ nor C⊥ orthogonal since neither the lower right (2×2) block of X0′X0 nor the centered Z′Z is diagonal. Moreover, the uncentered VIFus as listed in (3) are not the vaunted relative increases in variances owing to nonorthogonal columns of X0. Indeed, the only opportunity for V⊥-orthogonality here is between columns [X1,X2].Nonetheless, from Section4.1 we utilize Theorem 4(i) and (10) to connect the collinearity indices of [9] to angles between subspaces. Specifically, Minitab recovered values for the residuals {RSj=∥xj-Pjxj∥2;j=0,1,2}; further computations proceed routinely as listed in Table 1. In particular, the principal angle θ0 between the constant vector and the span of the regressor vectors is θ0=31.751deg, as anticipated in Remark 7.Table 1
Values for residualsRSj=∥xj-Pjxj∥2, for  ∥Xj∥2, for κuj2=VIFu(βj^), for (1-1/κuj2)1/2, and for θj, for j=0,1,2.
j
R
S
j
∥
X
j
∥
2
κ
j
2
(
1
-
1
/
κ
u
j
2
)
1
/
2
θ
j (deg)
0
1.3846
5.0
3.6111
0.8503
31.751
1
0.6429
2.5
3.8889
0.8619
30.470
2
2.5714
3.0
1.1667
0.3780
67.792


## 5.2.V⊥-Orthogonality

For subsequent reference letD(x) be a design as in (1) but with second-moment matrix:
(18)U(x)=[53132.5x1x3].
Invoking Definition 8, we seek a V⊥-orthogonal Reference with moment matrix U(0) having X1′X2=0,thatis,D(0). This is found constructively on retaining [15,•,X2] in X0, but replacing X1 by X10=[1,0.5,0.5,1,0]′, as listed in Table 2, giving the design D(0) with columns [15,X10,X2] as Reference.Table 2
Designs{D(x);  x∈[-0.5,0.0,0.5,0.6,1.0,1.5]}, obtained by substituting X10 = [X11,X12,X13,X14,X15]′ in [15, •, X2] of order (5×3), and corresponding variances {Var(β^0),Var(β^1),Var(β^2)}.
Design
X
11
X
12
X
13
X
14
X
15
Var
(
β
^
0
)
Var
(
β
^
1
)
Var
(
β
^
2
)
D
(
-
0.5
)
1.0
0.0
0.5
1.0
0.5
1.9333
3.7333
0.9333
D
(
0.0
)
1.0
0.5
0.5
1.0
0.0
0.9375
1.7500
0.4375
D
(
0.5
)
0.5
1.0
0.0
1.0
0.5
0.7436
1.4359
0.3590
D
(
0.6
)
0.6091
0.5793
0.6298
1.1819
0.000
0.7286
1.4286
0.3570
D
(
1.0
)
0.0
0.5
0.5
1.0
1.0
0.7222
1.5556
0.3889
D
(
1.5
)
0.0
1.0
0.5
0.5
1.0
0.9130
2.4348
0.6087Accordingly,RV=U(0) is “as V⊥-orthogonal as it can get,” given the lengths and sums of (X1,X2) as prescribed in the experiment and as preserved in the Reference model. Lemma 9, with X-=[0.6,0.2]′ and D= Diag(2.5,3), gives X-′D-1X-=0.157333 and γ=5/(1-5X-′D-1X-)=23.4375. Applications of Lemma 9(iii)-(iv) in succession give the reference variances
(19)Var(β^0∣X′X=D)=[5(1-5X-′D-1X-)]-1=0.9375,Var(β^1∣X′X=D)=1d11[1+γX-12d11]=25[1+0.72γ5]=1.7500,Var(β^2∣X′X=D)=1d22[1+γX-22d22]=13[1+0.04γ3]=0.4375.As Reference, these combine with actual variances fromV=[Vij] at (1),  giving  VFvfor  the original design  X0 relative to RV.For example, VFv(β^0) = 0.7222/0.9375 = 0.7704, with   V11=0.7222  from (1), and 0.9375 from (19). This contrasts with VIFu(β^0)=3.6111 in (3) as in Section 2.2 and Table 2, it is noteworthy that the nonorthogonal design D(1) yields uniformly smaller variances than the V⊥-orthogonal RV, namely, D(0).Further versions{D(x);  x∈[-0.5,0.0,0.5,0.6,1.0,1.5]} of our basic design are listed for comparison in Table 2, where x = X1′X2 for the various designs. The designs themselves are exhibited on transposing rows X1′= [X11,X12,X13,X14,X15] from Table 2, and substituting each into the template X00=[15,•,X2]. The design D(2) was constructed but not listed since its X0′X0 is not invertible. Clearly these are actual designs amenable to experimental deployment.


## 5.3.C⊥-Orthogonality

ContinuingX0=[15,X1,X2] and invoking Definition 12(ii), we seek a C⊥-orthogonal Reference having the matrix Cw as diagonal. This is identified as D(0.6) in Table 2. From this the matrix RC and its inverse are
(20)RC=[53132.50.610.63],RC-1=[0.7286-0.8572-0.0714-0.85711.42860.0-0.07140.00.3571].
The variance factors are listed in (4) where, for example, VFc(β^2) = 0.3889/0.3571 = 1.0889. As distinct from conventional VIFcs for β^1 and β^2 only, our VFc(β^0) here reflects Variance Deflation, wherein β0 is estimated more precisely in the initial C⊥-nonorthogonal design.Remark 15.
Observe that the choice forRC may be posed as seeking RC=U(x) such that RC-1(2,3)=0.0, then solving numerically for x using Maple for example. However, the algorithm in Definition 12(ii) affords a direct solution: that Cw should be diagonal stipulates its off-diagonal element as X1′X2-3/5 = 0, so that X1′X2 = 0.6 in RC at expression (20).To illustrate Theorem4(iii), we compute cos(θ) = (1-1/1.0889)1/2 = 0.2857 and θ = 73.398 deg as the angle between the vectors [X1,X2] of (1) when centered to their means. For slopes in M0, [9] shows that {VIFc(β^j)<VIFu(β^j);1≤i≤k}. This follows since their numerators are equal, but denominators are reciprocals of lengths of the centered and uncentered regressor vectors. To illustrate, numerators are equal for VIFu(β^3)=0.3889/(1/3)=1.1667 and for VIFc(β^3)=0.3889/(1/2.8)=1.0889, but denominators are reciprocals of X2′X2=3 and ∑j=15‍(X2j-X-2)2=2.8.A further note on orthogonality is germane. Suppose that the actual experiment isD(0) yet, towards a thorough diagnosis, the user evaluates the conventional VIFc(β^1)=1.2250=VIFc(β^2) as ratios of variances of D(0) to D(0.6) from Table 2. Unfortunately, their meaning is obscured since a design cannot at once be V⊥ and C⊥-orthogonal.As in Remark2, we next alter the basic design X0 at (1) on shifting columns of the measurements to [0,0] as minima and scaling to have squared lengths equal to n=5. The resulting X0 follows directly from (1). The new matrix U=X0'X0 and its inverse V are
(21)U=[54.24264.24264.2426544.242645],V=[1-0.4714-0.4714-0.47140.7778-0.2222-0.4714-0.22220.7778]
giving conventional VIFus as [5,3.8889,3.8889]. Against C⊥-orthogonality, this gives the diagnostics
(22)[VFc(β^0),VFc(β^1),VFc(β^2)]=[0.8140,1.0889,1.0889]
demonstrating, in comparison with (4), that VFcs, apart from β^0, are invariant under translation and scaling, a consequence of Lemma 14.We further seek to compare variances for the shifted and scaled(X1,X2) against V⊥-orthogonality as Reference. However, from U=X0′X0 at (21) we determine that X-1=0.84853=X-2 and 5[(X-12/5)+(X-22/5)]=1.4400. Since the latter exceeds unity, we infer from Lemma 9(i) that V⊥-orthogonality is incompatible with this configuration of the data, so that the VFvs are undefined. Equivalently, on evaluating ϕ1=ϕ2=22.90deg and ϕ1+ϕ2=45.80<90deg, Corollary 10 asserts that RV is not positive definite. This appears to be anomalous, until we recall that V⊥-orthogonality is invariant under rescaling, but not recentering the regressors.


## 5.4. A Critique

We reiterate apparent ambiguity ascribed in the literature toorthogonality. A number of widely held guiding precepts has evolved, as enumerated in part in our opening paragraph and in Section 3. As in Remark 3, these clearly have been taken to apply verbatim for X0 and X0′X0 in M0, to be paraphrased in part as follows.(P1)
Ill-conditioning espouses inflated variances; that is,VIFs necessarily equal or exceed unity.(P2)
C⊥-orthogonal designs are “ideal” in that VIFcs for such designs are all unity; see [11] for example.We next reassess these precepts as they play out underV⊥ and C⊥ orthogonality in connection with Table 2.(C1)
For models inM0 having X-≠0, we reiterate that the uncentered VIFus at expression (3), namely [3.6111,3.8889,1.1667], overstate adverse effects on variances of the V⊥-nonorthogonal array, since X0′X0 cannot be diagonal. Moreover, these conventional values fail to discern that the revised values for VFvs at (5), namely, [0.7704,0.8889,0.8889], reflect that [β0,β1,β2] are estimated with greater efficiency in the V⊥-nonorthogonal array D(1).(C2)
ForV⊥-orthogonality, the claim P1 thus is false by counterexample. The VFs at (5) are Variance Deflation Factors for design D(1), where X1′X2=1, relative to the V⊥-orthogonal D(0).(C3)
Additionally, the variances{Var[β^0(x)], Var[β^1(x)], Var[β^2(x)]} in Table 2 are all seen to decrease from D(0):[0.9375, 1.7500, 0.4375] → D(1):[0.7222, 1.5556, 0.3889] despite the transition from the V⊥-orthogonal D(0) to the V⊥-nonorthogonal D(1). Similar trends are confirmed in other ill-conditioned data sets from the literature.(C4)
ForC⊥-orthogonal designs, the claim P2 is false by counterexample. Such designs need not be “ideal,” as β0 may be estimated more efficiently in a C⊥-nonorthogonal design, as demonstrated by the VFcs for β^0 at (4) and (22). Similar trends are confirmed elsewhere, as seen subsequently.(C5)
VFs for β^0 are critical in prediction, where prediction variances necessarily depend on Var(β^0), especially for predicting near the origin of the system of coordinates.(C6)
Dissonance betweenV⊥ and C⊥ is seen in Table 2, where D(0.6), as C⊥-orthogonal with X1′X2=0.6, is the antithesis of V⊥-orthogonality at D(0), where X1′X2=0.(C7)
In short, these transparent examples serve to dispel the decades-old mantra that ill-conditioning necessarily spawns inflated variances for models inM0, and they serve to illuminate the contributing structures.


## 6. Orthogonal and Linked Arrays

A genuineV⊥-orthogonal array was generated as eigenvectors E=[E1,E2,…,E8] from a positive definite (8×8) matrix (see Table 8.3 of [11, p.377]) using PROCIML of the SAS programming package. The columns [X1,X2,X3], as the second through fourth columns of Table 3, comprise the first three eigenvectors scaled to length 8. These apply for models in Mw and M0 to be analyzed. In addition, linked vectors (Z1,Z2,Z3) were constructed as Z1=c(0.9E1+0.1E2), Z2=c(0.9E2+0.1E3), and Z3=c(0.9E3+0.1E4), where c=8/0.82. Clearly these arrays, as listed in Table 3, are not archaic abstractions, as both are amenable to experimental implementation.Table 3
BasicV⊥-orthogonal design X0 = [18, X1, X2, X3], and a linked design Z0 = [18, Z1, Z2, Z3], of order (8×4).
Orthogonal(X1,X2,X3)
Linked(Z1,Z2,Z3)
1
8
X
1
X
2
X
3
1
8
Z
1
Z
2
Z
3
1
−1.084470
0.056899
0.541778
1
−1.071550
0.116381
0.653486
1
−1.090010
−0.040490
0.117026
1
−1.087820
−0.027320
0.116086
1
0.979050
0.002564
2.337454
1
0.973345
0.260677
2.199687
1
−1.090230
0.016839
0.170711
1
−1.081700
0.035588
0.070694
1
0.127352
2.821941
−0.071670
1
0.438204
2.796767
−0.070710
1
1.045409
−0.122670
−1.476870
1
1.025468
−0.285010
−1.618560
1
1.090803
−0.088970
0.043354
1
1.074306
−0.083640
0.176928
1
1.090739
−0.092300
0.108637
1
1.073875
−0.079740
0.244567


### 6.1. The ModelM0

We consider in turn the orthogonal and the linked series.


#### 6.1.1. Orthogonal Data

MatricesX0′X0 and (X0′X0)-1 for the orthogonal data under model M0 are listed in Table 4, where variances occupy diagonals of (X0′X0)-1. The conventional uncentered VIFus are [1.20296,1.02144,1.12256,1.05888]. Since X-≠0, we find the angle between the constant and the span of the regressor vectors to be θ0=65.748deg as in Theorem 4(ii). Moreover, the angle between X1 and the span of [1n,X2,X3], namely, θ1=81.670deg, is not 90deg because of collinearity with the constant vector, despite the mutual orthogonality of [X1,X2,X3]. Observe here that X0′X0 already is V⊥-orthogonal; accordingly, RV = X0′X0; and thus the VFvs are all unity.Table 4
MatricesX0′X0 and (X0′X0)−1 for the orthogonal data under model M0.
X
0
′
X
0
(
X
0
′
X
0
)
-
1
8
1.0686
2.5538
1.7704
0.1504
−0.0201
−0.0480
−0.0333
1.0686
8
0
0
−0.0201
0.1277
0.0064
0.0044
2.5538
0
8
0
−0.0480
0.0064
0.1403
0.0106
1.7704
0
0
8
−0.0333
0.0044
0.0106
0.1324In view of dissonance betweenV⊥ and C⊥ orthogonality, the sums of squares and products for the mean-centered X are
(23)X′BnX=[7.8572-0.3411-0.2365-0.34117.1847-0.5652-0.2365-0.56527.6082],
reflecting nonnegligible negative dependencies, to distinguish the V⊥-orthogonal X from the C⊥-nonorthogonal matrix Z=BnX of deviations.To continue, we suppose instead that theV⊥-orthogonal model was to be recast as C⊥-orthogonal. The Reference model and its inverse are listed in Table 5. Variance factors, as ratios of diagonal elements of (X0′X0)-1 in Table 4 to those of RC-1 in Table 5, are
(24)[VFc(β^0),VFc(β^1),VFc(β^2),VFc(β^3)]=[1.0168,1.0032,1.0082,1.0070],
reflecting negligible differences in precision. This parallels results reported in Table 2 comparing D(0) to D(0.6) as Reference, except for larger differences in Table 2, namely, [1.2868,1.2250,1.2250].Table 5
MatricesRC and RC-1 for checking C⊥-orthogonality in the orthogonal data X0 under model M0.
R
C
R
C
-
1
8
1.0686
2.5538
1.7704
0.1479
−0.0170
−0.0444
−0.0291
1.0686
8
0.3411
0.2365
−0.0170
0.1273
0
0
2.5538
0.3411
8
0.5652
−0.0444
0
0.1392
0
1.7704
0.2365
0.5652
8
−0.0291
0
0
0.1314


#### 6.1.2. Linked Data

For the Linked Data under modelM0, the matrix Z0′Z0 follows routinely from Table 3; its inverse (Z0′Z0)-1 is listed in Table 6. The conventional uncentered VIFus are now [1.20320,1.03408,1.13760,1.05480]. These again fail to gage variance inflation since Z0′Z0 cannot be diagonal. From (10) the principal angle between the constant vector and the span of the regressor vectors is θ0=65.735 degrees.Table 6
Matrices(Z0′Z0)-1 and RV-1 for checking V⊥-orthogonality for the linked data under model M0.
(
Z
0
′
Z
0
)
-
1
R
V
-
1
0.1504
−0.0202
−0.0461
−0.0283
0.1551
−0.0261
−0.0530
−0.0344
−0.0202
0.1293
−0.00797
0.0053
−0.0261
0.1294
0.0089
0.0058
−0.0461
−0.0079
0.1422
−0.0054
−0.0530
0.0089
0.1431
0.0117
−0.0283
0.0053
−0.0054
0.1319
−0.0344
0.0058
0.0117
0.1326(i)
V
⊥
Reference Model. To rectify this malapropism, we seek in Definition 8 a model RV as Reference. This is found on setting all off-diagonal elements of Z0′Z0 to zero, excluding the first row and column. Its inverse RV-1 is listed in Table 6. The VFvs are ratios of diagonal elements of (Z0′Z0)-1 on the left to diagonal elements of RV-1 on the right, namely,
(25)[VFv(β^0),VFv(β^1),VFv(β^2),VFv(β^3)]=[0.9697,0.9991,0.9936,0.9943].
Against conventional wisdom, it is counter-intuitive that [β0,β1,β2,β3] are all estimated with greater precision in the V⊥-nonorthogonal model Z0′Z0 than in the V⊥-orthogonal model RV of Definition 8. As in Section 5.4, this serves again to refute the tenet that ill-conditioning espouses inflated variances for models in M0, that is, that VFs necessarily equal or exceed unity.(ii)
C
⊥
Reference Model. Further consider variances, were the Linked Data to be centered. As in Definition 12(ii), we seek a Reference model RC such that Cw=(W-M) is diagonal. The required RC and its inverse are listed in Table 7. Since variances in the Linked Data are [0.15040,0.12926,0.14220,0.13185] from Table 6 and Reference values appear as diagonal elements of RC-1 on the right in Table 7, their ratios give the centered VFcs as
(26)[VFc(β^0),VFc(β^1),VFc(β^2),VFc(β^3)]=[0.9920,1.0049,1.0047,1.0030].
We infer that changes in precision would be negligible, if instead the Linked Data experiment was to be recast as a C⊥-orthogonal experiment.Table 7
MatricesRC and RC-1 for checking C⊥-orthogonality in the linked data under model M0.
R
C
R
C
-
1
8
1.3441
2.7337
1.7722
0.1516
−0.0216
−0.0484
−0.0291
1.3441
8
0.4593
0.2977
−0.0216
0.1286
0
0
2.7337
0.4593
8
0.6056
−0.0484
0
0.1415
0
1.7722
0.2977
0.6056
8
−0.0291
0
0
0.1314As in Remark15, the reference matrix RC is constrained by the first and second moments from X0′X0 and has free parameters {x1,x2,x3} for the {(2,3),(2,4),(3,4)} entries. The constraints for C⊥-orthogonality are {RC-1(2,3)=RC-1(2,4)=RC-1(3,4)=0}. Although this system of equations can be solved with numerical software such as Maple, the algorithm stated in Definition 12 easily yields the direct solution given here.


### 6.2. The ModelMw

For the Linked Data take{Yi=β1Zi1+β2Zi2+β3Zi3+ϵi;1≤i≤8} as {Y=Zβ+ϵ}, where the expected response at {Z1=Z2=Z3=0} is E(Y)=0.0, and there is no occasion to translate the regressors. The lower right (3×3) submatrix of Z0′Z0(4×4) from M0 is Z′Z; its inverse is listed in Table 8. The ratios of diagonal elements of (Z′Z)-1 to reciprocals of diagonal elements of Z′Z are the conventional uncentered VIFus, namely, [1.0123,1.0247,1.0123]. Equivalently, scaling Z′Z to have unit diagonals and inverting gives C-1 as in Table 8; its diagonal elements are the VIFus. Throughout Section 6, these comprise the only correct interpretation of conventional VIFus as genuine variance ratios.Table 8
Matrices (Z′Z)−1 and C-1 for the linked data under model Mw.
(
Z
′
Z
)
-
1
C
-
1
0.1265
−0.0141
0.0015
1.0123
−0.1125
0.0123
−0.0141
0.1281
−0.0141
−0.1125
1.0247
−0.1125
0.0015
−0.0141
0.1265
0.0123
−0.1125
1.0123


## 6.1. The ModelM0

We consider in turn the orthogonal and the linked series.


### 6.1.1. Orthogonal Data

MatricesX0′X0 and (X0′X0)-1 for the orthogonal data under model M0 are listed in Table 4, where variances occupy diagonals of (X0′X0)-1. The conventional uncentered VIFus are [1.20296,1.02144,1.12256,1.05888]. Since X-≠0, we find the angle between the constant and the span of the regressor vectors to be θ0=65.748deg as in Theorem 4(ii). Moreover, the angle between X1 and the span of [1n,X2,X3], namely, θ1=81.670deg, is not 90deg because of collinearity with the constant vector, despite the mutual orthogonality of [X1,X2,X3]. Observe here that X0′X0 already is V⊥-orthogonal; accordingly, RV = X0′X0; and thus the VFvs are all unity.Table 4
MatricesX0′X0 and (X0′X0)−1 for the orthogonal data under model M0.
X
0
′
X
0
(
X
0
′
X
0
)
-
1
8
1.0686
2.5538
1.7704
0.1504
−0.0201
−0.0480
−0.0333
1.0686
8
0
0
−0.0201
0.1277
0.0064
0.0044
2.5538
0
8
0
−0.0480
0.0064
0.1403
0.0106
1.7704
0
0
8
−0.0333
0.0044
0.0106
0.1324In view of dissonance betweenV⊥ and C⊥ orthogonality, the sums of squares and products for the mean-centered X are
(23)X′BnX=[7.8572-0.3411-0.2365-0.34117.1847-0.5652-0.2365-0.56527.6082],
reflecting nonnegligible negative dependencies, to distinguish the V⊥-orthogonal X from the C⊥-nonorthogonal matrix Z=BnX of deviations.To continue, we suppose instead that theV⊥-orthogonal model was to be recast as C⊥-orthogonal. The Reference model and its inverse are listed in Table 5. Variance factors, as ratios of diagonal elements of (X0′X0)-1 in Table 4 to those of RC-1 in Table 5, are
(24)[VFc(β^0),VFc(β^1),VFc(β^2),VFc(β^3)]=[1.0168,1.0032,1.0082,1.0070],
reflecting negligible differences in precision. This parallels results reported in Table 2 comparing D(0) to D(0.6) as Reference, except for larger differences in Table 2, namely, [1.2868,1.2250,1.2250].Table 5
MatricesRC and RC-1 for checking C⊥-orthogonality in the orthogonal data X0 under model M0.
R
C
R
C
-
1
8
1.0686
2.5538
1.7704
0.1479
−0.0170
−0.0444
−0.0291
1.0686
8
0.3411
0.2365
−0.0170
0.1273
0
0
2.5538
0.3411
8
0.5652
−0.0444
0
0.1392
0
1.7704
0.2365
0.5652
8
−0.0291
0
0
0.1314


### 6.1.2. Linked Data

For the Linked Data under modelM0, the matrix Z0′Z0 follows routinely from Table 3; its inverse (Z0′Z0)-1 is listed in Table 6. The conventional uncentered VIFus are now [1.20320,1.03408,1.13760,1.05480]. These again fail to gage variance inflation since Z0′Z0 cannot be diagonal. From (10) the principal angle between the constant vector and the span of the regressor vectors is θ0=65.735 degrees.Table 6
Matrices(Z0′Z0)-1 and RV-1 for checking V⊥-orthogonality for the linked data under model M0.
(
Z
0
′
Z
0
)
-
1
R
V
-
1
0.1504
−0.0202
−0.0461
−0.0283
0.1551
−0.0261
−0.0530
−0.0344
−0.0202
0.1293
−0.00797
0.0053
−0.0261
0.1294
0.0089
0.0058
−0.0461
−0.0079
0.1422
−0.0054
−0.0530
0.0089
0.1431
0.0117
−0.0283
0.0053
−0.0054
0.1319
−0.0344
0.0058
0.0117
0.1326(i)
V
⊥
Reference Model. To rectify this malapropism, we seek in Definition 8 a model RV as Reference. This is found on setting all off-diagonal elements of Z0′Z0 to zero, excluding the first row and column. Its inverse RV-1 is listed in Table 6. The VFvs are ratios of diagonal elements of (Z0′Z0)-1 on the left to diagonal elements of RV-1 on the right, namely,
(25)[VFv(β^0),VFv(β^1),VFv(β^2),VFv(β^3)]=[0.9697,0.9991,0.9936,0.9943].
Against conventional wisdom, it is counter-intuitive that [β0,β1,β2,β3] are all estimated with greater precision in the V⊥-nonorthogonal model Z0′Z0 than in the V⊥-orthogonal model RV of Definition 8. As in Section 5.4, this serves again to refute the tenet that ill-conditioning espouses inflated variances for models in M0, that is, that VFs necessarily equal or exceed unity.(ii)
C
⊥
Reference Model. Further consider variances, were the Linked Data to be centered. As in Definition 12(ii), we seek a Reference model RC such that Cw=(W-M) is diagonal. The required RC and its inverse are listed in Table 7. Since variances in the Linked Data are [0.15040,0.12926,0.14220,0.13185] from Table 6 and Reference values appear as diagonal elements of RC-1 on the right in Table 7, their ratios give the centered VFcs as
(26)[VFc(β^0),VFc(β^1),VFc(β^2),VFc(β^3)]=[0.9920,1.0049,1.0047,1.0030].
We infer that changes in precision would be negligible, if instead the Linked Data experiment was to be recast as a C⊥-orthogonal experiment.Table 7
MatricesRC and RC-1 for checking C⊥-orthogonality in the linked data under model M0.
R
C
R
C
-
1
8
1.3441
2.7337
1.7722
0.1516
−0.0216
−0.0484
−0.0291
1.3441
8
0.4593
0.2977
−0.0216
0.1286
0
0
2.7337
0.4593
8
0.6056
−0.0484
0
0.1415
0
1.7722
0.2977
0.6056
8
−0.0291
0
0
0.1314As in Remark15, the reference matrix RC is constrained by the first and second moments from X0′X0 and has free parameters {x1,x2,x3} for the {(2,3),(2,4),(3,4)} entries. The constraints for C⊥-orthogonality are {RC-1(2,3)=RC-1(2,4)=RC-1(3,4)=0}. Although this system of equations can be solved with numerical software such as Maple, the algorithm stated in Definition 12 easily yields the direct solution given here.


## 6.1.1. Orthogonal Data

MatricesX0′X0 and (X0′X0)-1 for the orthogonal data under model M0 are listed in Table 4, where variances occupy diagonals of (X0′X0)-1. The conventional uncentered VIFus are [1.20296,1.02144,1.12256,1.05888]. Since X-≠0, we find the angle between the constant and the span of the regressor vectors to be θ0=65.748deg as in Theorem 4(ii). Moreover, the angle between X1 and the span of [1n,X2,X3], namely, θ1=81.670deg, is not 90deg because of collinearity with the constant vector, despite the mutual orthogonality of [X1,X2,X3]. Observe here that X0′X0 already is V⊥-orthogonal; accordingly, RV = X0′X0; and thus the VFvs are all unity.Table 4
MatricesX0′X0 and (X0′X0)−1 for the orthogonal data under model M0.
X
0
′
X
0
(
X
0
′
X
0
)
-
1
8
1.0686
2.5538
1.7704
0.1504
−0.0201
−0.0480
−0.0333
1.0686
8
0
0
−0.0201
0.1277
0.0064
0.0044
2.5538
0
8
0
−0.0480
0.0064
0.1403
0.0106
1.7704
0
0
8
−0.0333
0.0044
0.0106
0.1324In view of dissonance betweenV⊥ and C⊥ orthogonality, the sums of squares and products for the mean-centered X are
(23)X′BnX=[7.8572-0.3411-0.2365-0.34117.1847-0.5652-0.2365-0.56527.6082],
reflecting nonnegligible negative dependencies, to distinguish the V⊥-orthogonal X from the C⊥-nonorthogonal matrix Z=BnX of deviations.To continue, we suppose instead that theV⊥-orthogonal model was to be recast as C⊥-orthogonal. The Reference model and its inverse are listed in Table 5. Variance factors, as ratios of diagonal elements of (X0′X0)-1 in Table 4 to those of RC-1 in Table 5, are
(24)[VFc(β^0),VFc(β^1),VFc(β^2),VFc(β^3)]=[1.0168,1.0032,1.0082,1.0070],
reflecting negligible differences in precision. This parallels results reported in Table 2 comparing D(0) to D(0.6) as Reference, except for larger differences in Table 2, namely, [1.2868,1.2250,1.2250].Table 5
MatricesRC and RC-1 for checking C⊥-orthogonality in the orthogonal data X0 under model M0.
R
C
R
C
-
1
8
1.0686
2.5538
1.7704
0.1479
−0.0170
−0.0444
−0.0291
1.0686
8
0.3411
0.2365
−0.0170
0.1273
0
0
2.5538
0.3411
8
0.5652
−0.0444
0
0.1392
0
1.7704
0.2365
0.5652
8
−0.0291
0
0
0.1314


## 6.1.2. Linked Data

For the Linked Data under modelM0, the matrix Z0′Z0 follows routinely from Table 3; its inverse (Z0′Z0)-1 is listed in Table 6. The conventional uncentered VIFus are now [1.20320,1.03408,1.13760,1.05480]. These again fail to gage variance inflation since Z0′Z0 cannot be diagonal. From (10) the principal angle between the constant vector and the span of the regressor vectors is θ0=65.735 degrees.Table 6
Matrices(Z0′Z0)-1 and RV-1 for checking V⊥-orthogonality for the linked data under model M0.
(
Z
0
′
Z
0
)
-
1
R
V
-
1
0.1504
−0.0202
−0.0461
−0.0283
0.1551
−0.0261
−0.0530
−0.0344
−0.0202
0.1293
−0.00797
0.0053
−0.0261
0.1294
0.0089
0.0058
−0.0461
−0.0079
0.1422
−0.0054
−0.0530
0.0089
0.1431
0.0117
−0.0283
0.0053
−0.0054
0.1319
−0.0344
0.0058
0.0117
0.1326(i)
V
⊥
Reference Model. To rectify this malapropism, we seek in Definition 8 a model RV as Reference. This is found on setting all off-diagonal elements of Z0′Z0 to zero, excluding the first row and column. Its inverse RV-1 is listed in Table 6. The VFvs are ratios of diagonal elements of (Z0′Z0)-1 on the left to diagonal elements of RV-1 on the right, namely,
(25)[VFv(β^0),VFv(β^1),VFv(β^2),VFv(β^3)]=[0.9697,0.9991,0.9936,0.9943].
Against conventional wisdom, it is counter-intuitive that [β0,β1,β2,β3] are all estimated with greater precision in the V⊥-nonorthogonal model Z0′Z0 than in the V⊥-orthogonal model RV of Definition 8. As in Section 5.4, this serves again to refute the tenet that ill-conditioning espouses inflated variances for models in M0, that is, that VFs necessarily equal or exceed unity.(ii)
C
⊥
Reference Model. Further consider variances, were the Linked Data to be centered. As in Definition 12(ii), we seek a Reference model RC such that Cw=(W-M) is diagonal. The required RC and its inverse are listed in Table 7. Since variances in the Linked Data are [0.15040,0.12926,0.14220,0.13185] from Table 6 and Reference values appear as diagonal elements of RC-1 on the right in Table 7, their ratios give the centered VFcs as
(26)[VFc(β^0),VFc(β^1),VFc(β^2),VFc(β^3)]=[0.9920,1.0049,1.0047,1.0030].
We infer that changes in precision would be negligible, if instead the Linked Data experiment was to be recast as a C⊥-orthogonal experiment.Table 7
MatricesRC and RC-1 for checking C⊥-orthogonality in the linked data under model M0.
R
C
R
C
-
1
8
1.3441
2.7337
1.7722
0.1516
−0.0216
−0.0484
−0.0291
1.3441
8
0.4593
0.2977
−0.0216
0.1286
0
0
2.7337
0.4593
8
0.6056
−0.0484
0
0.1415
0
1.7722
0.2977
0.6056
8
−0.0291
0
0
0.1314As in Remark15, the reference matrix RC is constrained by the first and second moments from X0′X0 and has free parameters {x1,x2,x3} for the {(2,3),(2,4),(3,4)} entries. The constraints for C⊥-orthogonality are {RC-1(2,3)=RC-1(2,4)=RC-1(3,4)=0}. Although this system of equations can be solved with numerical software such as Maple, the algorithm stated in Definition 12 easily yields the direct solution given here.


## 6.2. The ModelMw

For the Linked Data take{Yi=β1Zi1+β2Zi2+β3Zi3+ϵi;1≤i≤8} as {Y=Zβ+ϵ}, where the expected response at {Z1=Z2=Z3=0} is E(Y)=0.0, and there is no occasion to translate the regressors. The lower right (3×3) submatrix of Z0′Z0(4×4) from M0 is Z′Z; its inverse is listed in Table 8. The ratios of diagonal elements of (Z′Z)-1 to reciprocals of diagonal elements of Z′Z are the conventional uncentered VIFus, namely, [1.0123,1.0247,1.0123]. Equivalently, scaling Z′Z to have unit diagonals and inverting gives C-1 as in Table 8; its diagonal elements are the VIFus. Throughout Section 6, these comprise the only correct interpretation of conventional VIFus as genuine variance ratios.Table 8
Matrices (Z′Z)−1 and C-1 for the linked data under model Mw.
(
Z
′
Z
)
-
1
C
-
1
0.1265
−0.0141
0.0015
1.0123
−0.1125
0.0123
−0.0141
0.1281
−0.0141
−0.1125
1.0247
−0.1125
0.0015
−0.0141
0.1265
0.0123
−0.1125
1.0123


## 7. Case Study: Body Fat Data

### 7.1. The Setting

Body fat and morphogenic measurements were reported forn=20 human subjects in Table D.11 of [29, p.717]. Response and regressor variables are Y: amount of body fat; X1: triceps skinfold thickness; X2: thigh circumference; and X3: mid-arm circumference, under M0:{Yi=β0+β1X1+β2X2+β3X3+ϵi}. From the original data as reported the condition number of X0′X0 is 1,039,931, and the VIFus are [271.4800,446.9419,63.0998,77.8703]. On scaling columns of X to equal column lengths {∥Xi∥=5.0;i=1,2,3}, the resulting condition number of the revised X0′X0 is 2, 969.6518, and the uncentered VIFus are as before from invariance under scaling.Against complaints that regressors are remote from their natural origins, we center regressors to[0,0,0] as origin on subtracting the minimal element from each column as in Remark 2, and scaling these to {∥Xi∥=5.0;i=1,2,3}. This is motivated on grounds that centered diagnostics, apart from VIFc(β^0), are invariant under shifts and scalings from Lemma 14. In addition, under the new origin [0,0,0], β0 assumes prominence as the baseline response against which changes due to regressors are to be gaged. The original data are available on the publisher’s web page for [29].Taking these shifted and scaled vectors as columns ofX=[X1,X2,X3] in the revised X0=[1n,X], values for X0′X0 and its inverse are listed in Table 9. The condition number of X0′X0 is now 113.6969 and its VIFus are [6.7756,17.9987,4.2782,17.4484]. Additionally, from Theorem 4(ii) we recover the angle θ0=22.592deg to quantify collinearity of regressors with the constant vector.Table 9
MatricesX0′X0 and (X0′X0)-1 for the body fat data under model M0 centered to minima and scaled to equal lengths.
X
0
′
X
0
(
X
0
′
X
0
)
-
1
20
19.4365
19.4893
19.2934
0.3388
−0.1284
−0.1482
−0.0203
19.4365
25
19.4533
24.2362
−0.1284
0.7199
0.0299
−0.6224
19.4893
19.4533
25
19.6832
−0.1482
0.0299
0.1711
−0.0494
19.2934
24.2362
19.6832
25
−0.0203
−0.6224
−0.0494
0.6979In addition, the scaled and centered correlation matrix forX is
(27)R=[1.00.08470.87810.08471.00.14240.87810.14241.0]
indicating strong linkage between mean-centered versions of (X1,X3). Moreover, the experimental design is neither V⊥ nor C⊥-orthogonal, since neither the lower right (3×3) block of X0′X0 nor the centered matrix C=(X′X-nX-X-′) is diagonal.(i)
V
⊥
Reference Model. To compare this with a V⊥-orthogonal design, we turn again to Definition 8. However, evaluating the test criterion in Lemma 9(i) shows that this configuration is not commensurate with V⊥-orthogonality, so that the VFvs are undefined. Comparisons with C⊥-orthogonal models are addressed next.(ii)
C
⊥
Reference Model. As in Definition 12(ii), the centering matrix is
(28)nX-X-′=[18.888918.940218.749918.940218.991618.800718.749918.800718.6118].To check against C⊥-orthogonality, we obtain the matrix W of Definition 12(ii) on replacing off-diagonal elements of the lower right (3×3) submatrix of X0′X0 by corresponding off-diagonal elements of nX-X-′. The result is the matrix RC and its inverse as given in Table 10, where diagonal elements of RC-1 are the Reference variances. The VFcs, found as ratios of diagonal elements of (X0′X0)-1 relative to those of RC-1, are listed in Table 12 under 𝔾=[0,0,0], the indicator of Definition 12(i) for this case.Table 10
MatricesRC and RC-1 for the body fat data, adjusted to give off-diagonal elements the value 0.0, with code 𝔾=[0,0,0].
R
C
R
C
-
1
20
19.4365
19.4893
19.2934
0.5083
−0.1590
−0.1622
−0.1510
19.4365
25
18.9402
18.7499
−0.1590
0.1636
0
0
19.4893
18.9402
25
18.8007
−0.1622
0
0.1664
0
19.2934
18.7499
18.8007
25
−0.1510
0
0
0.1565


### 7.2. Variance Factors and Linkage

Traditional gages of ill-conditioning are patently absurd on occasion. By conventionVIFs are “all or none,” wherein Reference models entail strictly diagonal components. In practice some regressors are inextricably linked: to get, or even visualize, orthogonal regressors may go beyond feasible experimental ranges, require extrapolation beyond practical limits, and challenge credulity. Response functions so constrained are described in [30] as “picket fences.” In such cases, taking C to be diagonal as its C⊥ Reference is moot, at best an academic folly, abetted in turn by default in standard software packages. In short, conventional diagnostics here offer answers to irrelevant questions. Given pairs of regressors irrevocably linked, we seek instead to assess effects on variances for other pairs that could be unlinked by design. These comments apply especially to entries under 𝔾=[0,0,0] in Table 12.To proceed, we defineessential ill-conditioning as regressors inextricably linked, necessarily to remain so and as nonessential if regressors could be unlinked by design. As a followup to Section 3.2, for X-≠0 we infer that the constant vector is inextricably linked with regressors, thus accounting for essential ill-conditioning not removed by centering, contrary to claims in [4]. Indeed, this is the essence of Definition 8.Fortunately, these limitations may be remedied through Reference models adapted to this purpose. This in turn exploits the special notation and trappings of Definition12(i) as follows. In addition to 𝔾=[0,0,0], we iterate for indicators 𝔾=[g12,g13,g23] taking values in {[1,0,0], [0,1,0], [0,0,1], [1,1,0], [1,0,1], [0,1,1]}. Values of VFcs thus obtained are listed in Table 12. To fix ideas, the Reference RC for 𝔾=[1,0,0], that is, for the constraint RC(2,3)=X0′X0(2,3)=19.4533, is given in Table 11, together with its inverse. Found as ratios of diagonal elements of (X0′X0)-1 relative to those of RC-1 in Table 11, VFcs are listed in Table 12 under 𝔾=[1,0,0]. In short, RC is “as C⊥-orthogonal as it could be” given the constraint. Other VFcs in Table 12 proceed similarly. For all cases in Table 12, excluding 𝔾=[0,1,1], β0 is estimated with greater efficiency in the original model than any of the postulated Reference models.Table 11
MatricesRC and RC-1 for the body fat data, adjusted to give off-diagonal elements the value 0.0 except X1′X2
code 𝔾=[1,0,0].
R
C
R
C
-
1
20
19.4365
19.4893
19.2934
0.4839
−0.1465
−0.1497
−0.1510
19.4365
25
19.4533
18.7499
−0.1465
0.1648
−0.0141
0
19.4893
19.4533
25
18.8007
−0.1497
−0.0141
0.1676
0
19.2934
18.7499
18.8007
25
−0.1510
0
0
0.1565Table 12
SummaryVFcs for the body fat data centered to minima and scaled to common lengths, identified with varying 𝔾=[g12,g13,g23] from Definition 12.
Elements of𝔾
V
F
cs
g
12
g
13
g
23
β
^
0
β
^
1
β
^
2
β
^
3
0
0
0
0.6665
4.3996
1.0282
4.4586
1
0
0
0.7002
4.3681
1.0208
4.4586
0
1
0
0.9196
1.0073
1.0282
1.0208
0
0
1
0.7201
4.3996
1.0073
4.3681
1
1
0
0.9848
1.0057
1.0208
1.0208
1
0
1
0.7595
4.3681
1.0003
4.3681
0
1
1
1.0248
1.0073
1.0073
1.0160As in Remark15, the reference matrix RC for 𝔾=[1,0,0] is constrained by X1′X2=19.4533, to be retained, whereas X1′X3=x1 and X2′X3=x2 are to be determined from {RC-1(2,4)=RC-1(3,4)=0}. Although this system can be solved numerically as noted, the algorithm stated in Definition 12 easily yields the direct solution given here.RecallX1: triceps skinfold thickness; X2: thigh circumference; and X3: mid-arm circumference; and from (27) that (X1,X3) are strongly linked. Invoking the cutoff rule [24] for VIFs in excess of 4.0, it is seen for 𝔾=[0,0,0] in Table 12 that VFc(β^1) and VFc(β^3) are excessive, where [X1,X2,X3] are forced to be mutually uncorrelated as Reference. Remarkably similar values are reported for 𝔾∈{[1,0,0],[0,0,1],[1,0,1]}, all gaged against intractable Reference models wherein (X1,X3) are postulated to be uncorrelated, however, incredibly.On the other hand,VFcs for 𝔾∈{[0,1,0],[1,1,0],[0,1,1]} in Table 12 are likewise comparable, where (X1,X3) are allowed to remain linked at X1′X3 = 24.2362. Here the VFcs reflect negligible changes in efficiency of the estimates in comparison with the original X0′X0, where [X1,X2,X3] are all linked. In summary, negligible changes in overall efficiency would accrue on recasting the experiment so that (X1,X2) and (X2,X3) were pairwise uncorrelated.Table12 conveys further useful information on noting that VFs, as ratios, are multiplicative and divisible. For example, take the model coded 𝔾=[1,1,0], where X1′X2 and X1′X3 are retained at their initial values under X0′X0, namely, 19.4533 and 24.2362 from Table 9, with (X2,X3) unlinked. Now taking 𝔾=[0,1,0] as reference, we extract the VFs on dividing elements in Table 12 at 𝔾=[0,1,0], by those of 𝔾=[1,1,0]. The result is
(29)[VF(β^0),VF(β^1),VF(β^2),VF(β^3)]=[0.91960.9848,1.00731.0057,1.02821.0208,1.02081.0208]=[0.9338,1.0017,1.0072,1.0000].


## 7.1. The Setting

Body fat and morphogenic measurements were reported forn=20 human subjects in Table D.11 of [29, p.717]. Response and regressor variables are Y: amount of body fat; X1: triceps skinfold thickness; X2: thigh circumference; and X3: mid-arm circumference, under M0:{Yi=β0+β1X1+β2X2+β3X3+ϵi}. From the original data as reported the condition number of X0′X0 is 1,039,931, and the VIFus are [271.4800,446.9419,63.0998,77.8703]. On scaling columns of X to equal column lengths {∥Xi∥=5.0;i=1,2,3}, the resulting condition number of the revised X0′X0 is 2, 969.6518, and the uncentered VIFus are as before from invariance under scaling.Against complaints that regressors are remote from their natural origins, we center regressors to[0,0,0] as origin on subtracting the minimal element from each column as in Remark 2, and scaling these to {∥Xi∥=5.0;i=1,2,3}. This is motivated on grounds that centered diagnostics, apart from VIFc(β^0), are invariant under shifts and scalings from Lemma 14. In addition, under the new origin [0,0,0], β0 assumes prominence as the baseline response against which changes due to regressors are to be gaged. The original data are available on the publisher’s web page for [29].Taking these shifted and scaled vectors as columns ofX=[X1,X2,X3] in the revised X0=[1n,X], values for X0′X0 and its inverse are listed in Table 9. The condition number of X0′X0 is now 113.6969 and its VIFus are [6.7756,17.9987,4.2782,17.4484]. Additionally, from Theorem 4(ii) we recover the angle θ0=22.592deg to quantify collinearity of regressors with the constant vector.Table 9
MatricesX0′X0 and (X0′X0)-1 for the body fat data under model M0 centered to minima and scaled to equal lengths.
X
0
′
X
0
(
X
0
′
X
0
)
-
1
20
19.4365
19.4893
19.2934
0.3388
−0.1284
−0.1482
−0.0203
19.4365
25
19.4533
24.2362
−0.1284
0.7199
0.0299
−0.6224
19.4893
19.4533
25
19.6832
−0.1482
0.0299
0.1711
−0.0494
19.2934
24.2362
19.6832
25
−0.0203
−0.6224
−0.0494
0.6979In addition, the scaled and centered correlation matrix forX is
(27)R=[1.00.08470.87810.08471.00.14240.87810.14241.0]
indicating strong linkage between mean-centered versions of (X1,X3). Moreover, the experimental design is neither V⊥ nor C⊥-orthogonal, since neither the lower right (3×3) block of X0′X0 nor the centered matrix C=(X′X-nX-X-′) is diagonal.(i)
V
⊥
Reference Model. To compare this with a V⊥-orthogonal design, we turn again to Definition 8. However, evaluating the test criterion in Lemma 9(i) shows that this configuration is not commensurate with V⊥-orthogonality, so that the VFvs are undefined. Comparisons with C⊥-orthogonal models are addressed next.(ii)
C
⊥
Reference Model. As in Definition 12(ii), the centering matrix is
(28)nX-X-′=[18.888918.940218.749918.940218.991618.800718.749918.800718.6118].To check against C⊥-orthogonality, we obtain the matrix W of Definition 12(ii) on replacing off-diagonal elements of the lower right (3×3) submatrix of X0′X0 by corresponding off-diagonal elements of nX-X-′. The result is the matrix RC and its inverse as given in Table 10, where diagonal elements of RC-1 are the Reference variances. The VFcs, found as ratios of diagonal elements of (X0′X0)-1 relative to those of RC-1, are listed in Table 12 under 𝔾=[0,0,0], the indicator of Definition 12(i) for this case.Table 10
MatricesRC and RC-1 for the body fat data, adjusted to give off-diagonal elements the value 0.0, with code 𝔾=[0,0,0].
R
C
R
C
-
1
20
19.4365
19.4893
19.2934
0.5083
−0.1590
−0.1622
−0.1510
19.4365
25
18.9402
18.7499
−0.1590
0.1636
0
0
19.4893
18.9402
25
18.8007
−0.1622
0
0.1664
0
19.2934
18.7499
18.8007
25
−0.1510
0
0
0.1565


## 7.2. Variance Factors and Linkage

Traditional gages of ill-conditioning are patently absurd on occasion. By conventionVIFs are “all or none,” wherein Reference models entail strictly diagonal components. In practice some regressors are inextricably linked: to get, or even visualize, orthogonal regressors may go beyond feasible experimental ranges, require extrapolation beyond practical limits, and challenge credulity. Response functions so constrained are described in [30] as “picket fences.” In such cases, taking C to be diagonal as its C⊥ Reference is moot, at best an academic folly, abetted in turn by default in standard software packages. In short, conventional diagnostics here offer answers to irrelevant questions. Given pairs of regressors irrevocably linked, we seek instead to assess effects on variances for other pairs that could be unlinked by design. These comments apply especially to entries under 𝔾=[0,0,0] in Table 12.To proceed, we defineessential ill-conditioning as regressors inextricably linked, necessarily to remain so and as nonessential if regressors could be unlinked by design. As a followup to Section 3.2, for X-≠0 we infer that the constant vector is inextricably linked with regressors, thus accounting for essential ill-conditioning not removed by centering, contrary to claims in [4]. Indeed, this is the essence of Definition 8.Fortunately, these limitations may be remedied through Reference models adapted to this purpose. This in turn exploits the special notation and trappings of Definition12(i) as follows. In addition to 𝔾=[0,0,0], we iterate for indicators 𝔾=[g12,g13,g23] taking values in {[1,0,0], [0,1,0], [0,0,1], [1,1,0], [1,0,1], [0,1,1]}. Values of VFcs thus obtained are listed in Table 12. To fix ideas, the Reference RC for 𝔾=[1,0,0], that is, for the constraint RC(2,3)=X0′X0(2,3)=19.4533, is given in Table 11, together with its inverse. Found as ratios of diagonal elements of (X0′X0)-1 relative to those of RC-1 in Table 11, VFcs are listed in Table 12 under 𝔾=[1,0,0]. In short, RC is “as C⊥-orthogonal as it could be” given the constraint. Other VFcs in Table 12 proceed similarly. For all cases in Table 12, excluding 𝔾=[0,1,1], β0 is estimated with greater efficiency in the original model than any of the postulated Reference models.Table 11
MatricesRC and RC-1 for the body fat data, adjusted to give off-diagonal elements the value 0.0 except X1′X2
code 𝔾=[1,0,0].
R
C
R
C
-
1
20
19.4365
19.4893
19.2934
0.4839
−0.1465
−0.1497
−0.1510
19.4365
25
19.4533
18.7499
−0.1465
0.1648
−0.0141
0
19.4893
19.4533
25
18.8007
−0.1497
−0.0141
0.1676
0
19.2934
18.7499
18.8007
25
−0.1510
0
0
0.1565Table 12
SummaryVFcs for the body fat data centered to minima and scaled to common lengths, identified with varying 𝔾=[g12,g13,g23] from Definition 12.
Elements of𝔾
V
F
cs
g
12
g
13
g
23
β
^
0
β
^
1
β
^
2
β
^
3
0
0
0
0.6665
4.3996
1.0282
4.4586
1
0
0
0.7002
4.3681
1.0208
4.4586
0
1
0
0.9196
1.0073
1.0282
1.0208
0
0
1
0.7201
4.3996
1.0073
4.3681
1
1
0
0.9848
1.0057
1.0208
1.0208
1
0
1
0.7595
4.3681
1.0003
4.3681
0
1
1
1.0248
1.0073
1.0073
1.0160As in Remark15, the reference matrix RC for 𝔾=[1,0,0] is constrained by X1′X2=19.4533, to be retained, whereas X1′X3=x1 and X2′X3=x2 are to be determined from {RC-1(2,4)=RC-1(3,4)=0}. Although this system can be solved numerically as noted, the algorithm stated in Definition 12 easily yields the direct solution given here.RecallX1: triceps skinfold thickness; X2: thigh circumference; and X3: mid-arm circumference; and from (27) that (X1,X3) are strongly linked. Invoking the cutoff rule [24] for VIFs in excess of 4.0, it is seen for 𝔾=[0,0,0] in Table 12 that VFc(β^1) and VFc(β^3) are excessive, where [X1,X2,X3] are forced to be mutually uncorrelated as Reference. Remarkably similar values are reported for 𝔾∈{[1,0,0],[0,0,1],[1,0,1]}, all gaged against intractable Reference models wherein (X1,X3) are postulated to be uncorrelated, however, incredibly.On the other hand,VFcs for 𝔾∈{[0,1,0],[1,1,0],[0,1,1]} in Table 12 are likewise comparable, where (X1,X3) are allowed to remain linked at X1′X3 = 24.2362. Here the VFcs reflect negligible changes in efficiency of the estimates in comparison with the original X0′X0, where [X1,X2,X3] are all linked. In summary, negligible changes in overall efficiency would accrue on recasting the experiment so that (X1,X2) and (X2,X3) were pairwise uncorrelated.Table12 conveys further useful information on noting that VFs, as ratios, are multiplicative and divisible. For example, take the model coded 𝔾=[1,1,0], where X1′X2 and X1′X3 are retained at their initial values under X0′X0, namely, 19.4533 and 24.2362 from Table 9, with (X2,X3) unlinked. Now taking 𝔾=[0,1,0] as reference, we extract the VFs on dividing elements in Table 12 at 𝔾=[0,1,0], by those of 𝔾=[1,1,0]. The result is
(29)[VF(β^0),VF(β^1),VF(β^2),VF(β^3)]=[0.91960.9848,1.00731.0057,1.02821.0208,1.02081.0208]=[0.9338,1.0017,1.0072,1.0000].


## 8. Conclusions

Our goal is clarity in the actual workings ofVIFs as ill-conditioning diagnostics, thus to identify and rectify misconceptions regarding the use of VIFs for models X0=[1n,X] with intercept. Would-be arbiters for “correct” usage have divided between VIFcs for mean-centered regressors and VIFus for uncentered data. We take issue with both. Conventional but clouded vision holds that (i) VIFus gage variance inflation owing to nonorthogonal columns of X0 as vectors in ℝn; (ii) VIFs ≥1.0; and (iii) models having orthogonal mean-centered regressors are “ideal” in possessing VIFcs of value unity. Reprising Remark 3, these properties, widely and correctly understood for models in Mw, appear to have been expropriated without verification for models in M0 hence the anomalies.Accordingly, we have distinguished vector-space orthogonality (V⊥) of columns of  X0, in contrast to orthogonality (C⊥) of mean-centered columns of X. A key feature is the construction of second-moment arrays as Reference models (RV,RC), encoding orthogonalities of these types, and their Variance Factors as VFv’s and VFc’s, respectively. Contrary to convention, we demonstrate analytically and through case studies that (i′) VIFus do not gage variance inflation; (ii′) VFs ≥1.0 need not hold, whereas VF<1.0 represents Variance Deflation; and (iii′) models having orthogonal centered regressors are not necessarily “ideal.” In particular, variance deflation occurs when ill-conditioned data yield smaller variances than corresponding orthogonal surrogates.In short, our studies call out to adjudicate the prolonged dispute regarding centered and uncentered diagnostics for models inM0. Our summary conclusions follow.(S1)
The choice of a model, with or without intercept, is a substantive matter in the context of each experimental paradigm. That choice is fixed beforehand in a particular setting, is structural, and is beyond the scope of the present study. Snee and Marquardt [10] correctly assert that VIFs are fully credited and remain undisputed in models without intercept: these quantities unambiguously retain the meanings originally intended, namely, as ratios of variances to gage effects of nonorthogonal regressors. For models with intercept the choice is between centered and uncentered diagnostics, which is to be weighed as follows.(S2)
ConventionalVIFcs apply incompletely to slopes only, not necessarily grasping fully that a system is ill conditioned. Of particular concern is missing evidence on collinearity of regressors with the intercept. On the other hand, if C⊥-orthogonality pertains, then VFcs may supplant VIFcs as applicable to all of [β0,β1,…,βk]. Moreover, the latter may reveal that VFc(β^0)<1.0, as in cited examples and of special import in predicting near the origin of the coordinate system.(S3)
OurVFvs capture correctly the concept apparently intended by conventional VIFus. Specifically, if V⊥-orthogonality pertains, then VFvs, as genuine ratios of variances, may supplant VIFus as now discredited gages for variance inflation.(S4)
Returning to Section3.2, we subscribe fully, but for different reasons, to Belsley’s and others’ contention that uncentered VIFus are essential in assessing ill-conditioned data. In the geometry of ill-conditioning, these should be retained in the pantheon of regression diagnostics, not as now debunked gages of variance inflation, but as more compelling angular measures of the collinearity of each column of X0=[1n,X1,X2,…,Xk], with the subspace spanned by remaining columns.(S5)
Specifically, the degree of collinearity of regressors with the intercept is quantified byθ0deg which, if small, is known to “corrupt the estimates of all parameters in the model whether or not the intercept is itself of interest and whether or not the data have been centered” [21, p.90]. This in turn would appear to elevate θ0 in importance as a critical ill-conditioning diagnostic.(S6)
In contrast, Theorem4(iii) identifies conventional VIFcs with angles between a centered regressor and the subspace spanned by remaining centered regressors. For example,
(30)θ1=arccos([1-1VIFc(β^1)]1/2)
is the angle between the centered vector Z1=BnX1 and the remaining centered vectors. These lie in the linear subspace of ℝn comprising the orthogonal complement to 1n∈ℝn and thus bear on the geometry of ill-conditioning in this subspace.(S7)
Whereas conventionalVIFs are “all or none,” to be gaged against strictly diagonal components, our Reference models enable unlinking what may be unlinked, while leaving other pairs of regressors linked. This enables us to assess effects on variances attributed to allowable partial dependencies among some but not other regressors.In conclusion, the foregoing summary points to limitations in modern software packages.Minitab v.15, SPSS v.19, and SAS v.9.2 are standard software packages which compute collinearity diagnostics returning, with the default options, the centered VIFs {VIFc(β^j);1≤j≤k}. To compute the uncentered VIFus, one needs to define a variable for the constant term and perform a linear regression using the options of fitting without an intercept. On the other hand, computations for VFcs and VFvs, as introduced here, require additional if straightforward programming, for example, Maple and PROCIML of the SAS System, as utilized here.



---

*Source: 671204-2013-02-25.xml*