Condition Indexes and Variance Decompositions for Diagnosing

Collinearity in Linear Model Analysis of Survey Data 

Dan Liao1 and Richard Valliant2

1RTI International, 701 13th Street, N.W., Suite 750, Washington DC, 20005, dliao@rti.org 

2University of Michigan and University of Maryland, Joint Program in Survey Methodology, 

1218 Lefrak Hall, College Park, MD, 20742, rvalliant@survey.umd.edu 

Abstract 

Collinearities among explanatory variables in linear regression models affect estimates from survey data just as they do 

in non-survey data. Undesirable effects are unnecessarily inflated standard errors, spuriously low or high t-statistics, 

and parameter estimates with illogical signs. The available collinearity diagnostics are not generally appropriate for 

survey data because the variance estimators they incorporate do not properly account for stratification, clustering, 

and survey weights. In this article, we derive condition indexes and variance decompositions to diagnose collinearity 

problems in complex survey data. The adapted diagnostics are illustrated with data based on a survey of health 

characteristics. 

Keywords: diagnostics for survey data; multicollinearity; singular value decomposition; variance inflation. 

1 Introduction 

When predictor variables in a regression model are correlated with each other, this condition is referred to as collinearity. Undesirable side effects of collinearity are unnecessarily high standard errors, spuriously low or high t-statistics, 

and parameter estimates with illogical signs or ones that are overly sensitive to small changes in data values. In experimental design, it may be possible to create situations where the explanatory variables are orthogonal to each other, 

but this is not true with observational data. Belsley (1991) noted that: "... in nonexperimental sciences, ..., collinearity 

is a natural law in the data set resulting from the uncontrollable operations of the data-generating mechanism and is 

simply a painful and unavoidable fact of life." In many surveys, variables that are substantially correlated are collected for analysis. Few analysts of survey data have escaped the problem of collinearity in regression estimation, 

and the presence of this problem encumbers precise statistical explanation of the relationships between predictors and 

responses. 

Although many regression diagnostics have been developed for non-survey data, there are considerably fewer for 

survey data. The few articles that are available concentrate on identifying influential points and influential groups with 

abnormal data values or survey weights. Elliot (2007) developed Bayesian methods for weight trimming of linear and 

generalized linear regression estimators in unequal probability-of-inclusion designs. Li (2007a,b) and Li & Valliant 

(2011, 2009) extended a series of traditional diagnostic techniques to regression on complex survey data. Their papers 

cover residuals and leverages, several diagnostics based on case-deletion (DFBETA, DFBETAS, DFFIT, DFFITS, and 

Cook’s Distance), and the forward search approach. Although an extensive literature in applied statistics provides 

valuable suggestions and guidelines for data analysts to diagnose the presence of collinearity (e.g., Belsley et al. 1980; 

Belsley 1991; Farrar & Glauber 1967; Fox 1986; Theil 1971), almost none of this research touches upon diagnostics 

for collinearity when fitting models with survey data. One prior, survey-related paper on collinearity problems is (Liao 

& Valliant, 2010) which adapted variance inflation factors for linear models fitted with survey data. 

1 
Suppose the underlying structural model in the superpopulation is Y = XT β+e. The matrix X is an n×p matrix of 

predictors with n being the sample size; β is a p× 1 vector of parameters. The error terms in the model have a general 

variance structure e ∼ (0, σ2R) where σ

2

is an unknown constant and R is a unknown n × n covariance matrix. 

Define W to be the diagonal matrix of survey weights. We assume throughout that the survey weights are constructed 

in such a way that they can be used for estimating finite population totals. The survey weighted least squares (SWLS) 

estimator is 

βˆ

SW = (XTW X)

−1XTW Y ≡ A−1XTW Y ,

W−1

assuming A = XT X is invertible. Fuller (2002) describes the properties of this estimator. The estimator 

βˆ

SW is model unbiased for β under the model Y = XT β + e regardless of whether V arM(e) = σ

2R is specified 

correctly or not, and is approximately design-unbiased for the census parameter BU = (XU

T XU )

−1 XU

T Y U , in 

the finite population U of N units. The finite population values of the response vector and matrix of predictors are 

Y U = (Y1, ..., YN )

T

, and XU = (X1, ..., Xp) with Xk being the N × 1 vector of values for covariate k. 

The remainder of the paper is organized as follows. Section 2 reviews results on condition numbers and variance 

decompositions for ordinary least squares. These are extended to be appropriate for survey estimation in section 3. 

The fourth section gives some numerical illustrations of the techniques. Section 5 is a conclusion. In most derivations, 

we use model-based calculations since the forms of the model-variances are useful for understanding the effects of 

collinearity. However, when presenting variance decompositions, we use estimators that have both model- and designbased justifications. 

2 Condition Indexes and Variance Decompositions in Ordinary Least Squares Estimation 

In this section we briefly review techniques for diagnosing collinearity in ordinary least squares (OLS) estimation 

based on condition indexes and variance decompositions. These methods will be extended in section 3 to cover 

complex survey data. 

2.1 Eigenvalues and Eigenvectors of XT X

When there is an exact (perfect) collinear relation in the n × p data matrix X, we can find a set of values, v =

(v1, . . . , vp), not all zero, such that 

v1X1 + · · · + vpXp = 0, or Xv = 0. (1) 

However, in practice, when there exists no exact collinearity but some near dependencies in the data matrix, it may be 

possible to find one or more non-zero vectors v such that Xv = a with a 6= 0 but close to 0. Alternatively, we might 

say that a near dependency exists if the length of vector a, kak, is small. To normalize the problem of finding the set 

of v’s that makes kak small, we consider only v with unit length, that is, with kvk = 1. Belsley (1991) discusses 

the connection of the eigenvalues and eigenvectors of XT X with the normalized vector v and kak. The minimum 

length kak is simply the positive square root of the smallest eigenvalue of XT X. The v that produces the a with 

minimum length must be the eigenvector of XT X that corresponds to the smallest eigenvalue. As discussed in the 

next section, the eigenvalues and eigenvectors of X are related to those of XT X and have some advantages when 

examining collinearity. 

2.2 Singular-Value Decomposition, Condition Number and Condition Indexes 

The singular-value decomposition (SVD) of matrix X is very closely allied to the eigensystem of XT X, but with 

its own advantages. The n × p matrix X can be decomposed as X = UDV T

, where U

T U = VTV = Ip and 

D = diag(μ1, . . . , μp) is the diagonal matrix of singular values (or eigenvalues) of X. Here, the three components 

2 
in the decomposition are matrices with very special, highly exploitable properties: U is n × p (the same size as X) 

and is column orthogonal; V is p × p and both row and column orthogonal; D is p × p, nonnegative and diagonal. 

Belsley et al. (1980) felt that the SVD of X has several advantages over the eigen system of XT X, for the sake 

of both statistical usages and computational complexity. For prediction, X is the focus not the cross-product matrix 

XT X since Yˆ = Xβˆ. In addition, the lengths kak of the linear combinations (1) of X that are relate to collinearity 

are properly defined in terms of the square roots of the eigenvalues of XT X, which are the singular values of X. 

A secondary consideration, given current computing power, is that the singular value decomposition of X avoids the 

additional computational burden of forming XT X, an operation involving np2 unneeded sums and products, which 

may lead to unnecessary truncation error. 

The condition number of X is defined as κ(X) = μmax/μmin, where μmax and μmin are the maximum and minimum singular values of X. Condition indexes are defined as ηk = μmax/μk. The closer that μmin is to zero, the 

nearer XT X is to being singular. Empirically, if a value of κ or η exceeds a cutoff value of, say, 10 to 30, two or 

more columns of X have moderate or strong relations. The simultaneous occurrence of several large ηk’s is always 

remarkable for the existence of more than one near dependency. 

One issue with the SVD is whether the X’s should be centered around their means. Marquardt (1980) maintained 

that the centering of observations removes nonessential ill conditioning. In contrast, Belsley (1984) argues that meancentering typically masks the role of the constant term in any underlying near-dependencies. A typical case is a 

regression with dummy variables. For example, if gender is one of the independent variables in a regression and 

most of the cases are male (or female), then the dummy for gender can be strongly collinear with the intercept. The 

discussions following Belsley (1984) illustrate the differences of opinion that occur among practitioners (Wood, 1984; 

Snee & Marquardt, 1984; Cook, 1984). Moreover, in linear regression analysis, Wissmann et al. (2007) found that 

the degree of multicollinearity with dummy variables may be influenced by the choice of reference category. In this 

article, we do not center the X’s but will illustrate the effect of the choice of reference category in Section 4. 

Another problem with the condition number is that it is affected by the scale of the x measurements (Steward, 1987). 

By scaling down any column of X, the condition number can be made arbitrarily large. This situation is known as 

artificial ill-conditioning. Belsley (1991) suggests scaling each column of the design matrix X using the Euclidean 

norm of each column before computing the condition number. This method is implemented in SAS and the package 

perturb of the statistical software R (Hendrickx, 2010). Both use the root mean square of each column for scaling as its 

standard procedure. The condition number and condition indexes of the scaled matrix X are referred to as the scaled 

condition number and scaled condition indexes of the matrix X. Similarly, the variance decomposition proportions 

relevant to the scaled X (which will be discussed in next section) will be called the scaled variance decomposition 

proportions. 

2.3 Variance Decomposition Method 

To assess the extent to which near dependencies (i.e., having high condition indexes of X and XT X) degrade the 

estimated variance of each regression coefficient, Belsley et al. (1980) reinterpreted and extended the work of Silvey 

(1969) by decomposing a coefficient variance into a sum of terms each of which is associated with a singular value. In 

the remainder of this section, we review the results of ordinary least squares (OLS) under the model EM(Y ) = Xβ

and V arM(Y ) = σ

2In where In is the n×n identity matrix. These results will be extended to survey weighted least 

squares in section 3. Recall that the model variance-covariance matrix of the OLS estimator βˆ = (XT X)

−1XT Y

under the model with V arM(Y ) = σ

2In is V arM(βˆ) = σ2

(XT X)

−1

. Using the SVD, X = UDV T, V arM(βˆ)

can be written as: 

V arM(βˆ) = σ

2

[(UDV T)

T

(UDV T)]−1 = σ

2V D−2V

T

(2) 

and the k

th diagonal element in V arM(βˆ) is the estimated variance for the kth coefficient, βˆ

k. Using (2), V arM(βˆk)

3 
can be expressed as: 

2 vkj V ar(βˆ

k) = σ

2Σ

p

j=1 2

(3) 

μj

2 Σ

p where V = (vkj )p×p. Let φkj = vkj/μj

2

, φk = j=1φkj and Q = (φkj )p×p = (V D−1) · (V D−1), where ·

is the Hadamard (elementwise) product. The variance-decomposition proportions are πjk = φjk/φk, which is the 

proportion of the variance of the k

th regression coefficient associated with the jth component of its decomposition in 

−1  ̄ (3). Denote the variance decomposition proportion matrix as Π = (πjk)p×p = Q

T Q , where Q ̄ is the diagonal 

matrix with the row sums of Q on the main diagonal and 0 elsewhere. 

If the model is EM(Y ) = Xβ, V arM(Y ) = σ

2W−1

and weighted least squares is used, then 

βˆWLS = (XTW X)

−1XTW Y and V arM(βˆWLS) = σ2

(XTW X)

−1

. The decomposition in (3) holds with 

X ̃ W1/2 = X being decomposed as X ̃ = UDV T. However, in survey applications, it will virtually never be the 

case that the covariance matrix of Y is σ

2W−1

if W is the matrix of survey weights. Section 3 covers the more 

realistic case. 

In the variance decomposition (3), other things being equal, a small singular value μj can lead to a large component 

of V ar(βˆ

k). However, if vkj is small too, then V ar(βˆk) may not be affected by a small μj . One extreme case is when 

vkj = 0. Suppose the k

th and jth columns of X belong to separate orthogonal blocks. Let X ≡ [X1, X2] with 

XT

1 X2 = 0 and let the singular-value decompositions of X1 and X2 be given, respectively, as X1 = U1D11V

T

11

and X2 = U2D22V

T

22. Since U1 and U2 are the orthogonal bases for the space spanned by the columns of X1

and X2 respectively, XT

1 X2 = 0 implies U

T

1 U2 = 0 and U ≡ [U1, U2] is column orthogonal. The singular value 

decomposition of X is simply X = UDUT, with: 2

  D11 0 D = (4) 0 D22

and 

  V 11 0

V = . (5) 0 V 22

Thus V 12 = 0. An analogous result clearly applies to any number of mutually orthogonal subgroups. Hence, if all 

the columns in X are orthogonal, all the vkj = 0 when k =6 j and πkj = 0 likewise. When vkj is nonzero, this is a 

signal that predictors k and j are not orthogonal. 

Since at least one vkj must be nonzero in (3), this implies that a high proportion of any variance can be associated with 

a large singular value even when there is no collinearity. The standard approach is to check a high condition index 

associated with a large proportion of the variance of two or more coefficients when diagnosing collinearity, since there 

must be two or more columns of X involved to make a near dependency. Belsley et al. (1980) suggested showing the 

matrix Π and condition indexes of X in a variance decomposition table as below. If two or more elements in the j

th

row of matrix Π are relatively large and its associated condition index ηj is large too, it signals that near dependencies 

are influencing regression estimates. 

Condition Proportions of variance 

Index V arM(βˆ

1) V arM(βˆ2) · · · V arM(βˆp)

η1 π11 π12 · · · π1p

η2 π21 π22 · · · π2p

. . . . 

. . . . 

. . . . 

ηp πp1 πp2 · · · πpp

4 
3 Adaptation in Survey-Weighted Least Squares 

3.1 Condition Indexes and Variance Decomposition Proportions 

In survey-weighted least squares (SWLS), we are more interested in the collinear relations among the columns in the 

T

matrix X ̃ = W1/2X instead of X, since βˆ

SW = (X ̃ X ̃ )

−1X ̃ Y ̃ . Define the singular value decomposition of X ̃ to 

be X ̃ = UDV T

, where U, V , and D are usually different from the ones of X, due to the unequal survey weights. 

The condition number of X ̃ is defined as κ(X ̃ ) = μmax/μmin, where μmax and μmin are maximum and minimum 

singular values of X ̃ . The condition number of X ̃ is also usually different from the condition number of the data 

matrix X due to unequal survey weights. Condition indexes are defined as 

ηk = μmax/μk, k = 1, ..., p (6) 

where μk is one of the singular values of X ̃ . The scaled condition indexes and condition numbers are the condition 

 ̃ indexes and condition numbers of the scaled X. 

Based on the extrema of the ratio of quadratic forms (Lin, 1984), the condition number κ(X ̃ ) is bounded in the range 

of: 

1/2 1/2

w w min max

κ(X) ≤ κ(X ̃ ) ≤ κ(X), (7) 1/2 1/2

w w max min

where wmin and wmax are the minimum and maximum survey weights. This expression indicates that if the survey 

weights do not vary too much, the condition number in SWLS resembles the one in OLS. However, in a sample with 

a wide range of survey weights, the condition number can be very different between SWLS and OLS. When SWLS 

has a large condition number, OLS might not. In the case of exact linear dependence among the columns of X, the 

columns of X ̃ will also be linearly dependent. In this extreme case at least one eigenvalue of X will be zero, and both 

κ(X) and κ(X ̃ ) will be infinite. As in OLS, large values of κ or of the ηk’s of 10 or more may signal that two or more 

columns of X have moderate to strong dependencies. 

The model variance of the SWLS parameter estimator under a model with V arM(e) = σ

2R is: 

V arM(βˆ

SW ) = σ

2

(XTW X)

−1XTWRWX(XTW X)−1

(8) T

= σ

2

(X ̃ X ̃ )

−1G,

where 

G = (gij )p×p = XTWRWX(XTW X)

−1

(9) 

is the misspecification effect (MEFF) that represents the inflation factor needed to correct standard results for the effect 

of intracluster correlation in clustered survey data and for the fact that V arM(e) = σ

2R and not σ2W−1

(Scott & 

Holt, 1982). 

Using the SVD of X ̃ , we can rewrite V arM(βˆ

SW ) as 

V arM(βˆ

SW ) = σ

2V D−2V

T G. (10) 

The k

th diagonal element in V arM(βˆ) is the estimated variance for the kth coefficient, βˆ

k. Using (10), V arM(βˆk)

can be expressed as: 

V ar(βˆ

k) = σ

2Σ

p vkj (11) j=1 μ

2

λkj

j

where λkj = Σp

i=1vijgik. if R = W−1

, then G = Ip, λkj = vkj , and (11) reduces to (3). However, the situation is 

more complicated when G is not the identity matrix, i.e., when the complex design affects the variance of an estimated 

5 
regression coefficient. If predictors k and j are orthogonal, vkj = 0 for k 6= j and the variance in (11) depends only 

on the k

th singular value and is unaffected by gij ’s that are non-zero. If predictor k and several j’s are not orthogonal, 

then λkj has contributions from all of those eigenvectors and from the off-diagonal elements of the MEFF matrix G. 

The term λkj then measures both non-orthogonality of x’s and effects of the complex design. 

Consequently, we can define variance decomposition proportions and analogous to those for OLS but their interpretation is less straightforward. Let φkj = vkjλkj/μ2

j

, φk = Σp

j=1φkj and Q = (φkj )p×p = (V D−2

) · (V

T G)

T

. 

The variance-decomposition proportions are πjk = φjk/φk, which is the proportion of the variance of the kth regression coefficient associated with the j

th component of its decomposition in (11). Denote the variance decomposition 

proportion matrix as 

−1 Π = (πjk)p×p = Q

T Q ̄ , (12) 

where Q ̄ is the diagonal matrix with the row sums of Q on the main diagonal and 0 elsewhere. The interpretation of 

the proportions in (12) is not as clear-cut as for OLS because the effect of the MEFF matrix. Section 3.2 discusses the 

interpretation in more detail in the context of stratified cluster sampling. 

Analogous to the method for OLS regression, a variance decomposition table can be formed like the one at the end of 

section 2. When two or more independent variables are collinear (or “nearly dependent"), one singular value should 

make a large contribution to the variance of the parameter estimates associated with those variables. For example, if 

the proportions π31 and π32 for the variances of βˆ

SW1

and βˆ

SW2

are large, this would say that the third singular value 

makes a large contribution to both variances and that the first and second predictors in the regression are, to some 

extent, collinear. As shown in section 2.3, when the kth and jth columns in X are orthogonal, vkj = 0 and the jth 

singular value’s decomposition proportion πjk on V ar(βˆ

k) will be 0. 

Several special cases are worth noting. If R = W−1as assumed in WLS, then G = I. The variance decomposition 

in (11) has the same form as (2) in OLS. However, having R = W−1

in survey data would be unusual since survey 

weights are not typically computed based on the variance structure of a model. Note that V is still different from the 

one in OLS and is one component of the SVD of X ̃ instead of X. Another special case here is when R = I and the 

survey weights are equal, in which case the OLS results can be used. However, when the survey weights are unequal, 

even when R = I, the variance decomposition in (11) is different from (2) in OLS since G 6= I. In the next section, 

we will consider some special models that take the population features such as clusters and strata into account when 

estimating this variance decomposition. 

3.2 Variance Decomposition for A Model with Stratified Clustering 

The model variance of βˆ

SW in (8) contains the unknown R that must be estimated. In this section, we present 

an estimator for βˆ

SW that is appropriate for a model with stratified clustering. The variance estimator has both 

model-based and design-based justification. Suppose that in a stratified multistage sampling design, there are strata 

h = 1, ..., H in the population, clusters i = 1, ..., Nh in stratum h and units t = 1, ..., Mhi in cluster hi. We select 

clusters i = 1, ..., nh in stratum h and units t = 1, ..., mhi in cluster hi. Denote the set of sample clusters in stratum 

P

h

by sh and the sample of units in cluster hi as shi. The total number of sample units in stratum h is mh = i∈sh mhi, PH

and the total in the sample is m = h=1 mh. Assume that clusters are selected with varying probabilities and with 

replacement within strata and independently between strata. The model we consider is: 

T EM(Yhit) = xhitβ h = 1, . . . , H, i = 1, . . . , Nh, t = 1, . . . , Mhi

T = i0 CovM(εhit, εhi0t

0 ) = 0 where εhit = Yhit − xhitβ, i 6 (13) 

CovM(εhit, εh0i

0t0 ) = 0 h 6= h

0

.

6 
Units within each cluster are assumed to be correlated but the particular form of the covariances does not have to be 

specified for this analysis. The estimator βˆ

SW of the regression parameter can be written as: 

X X

H

T

βˆ

SW = (X ̃ X ̃ )

−1XT

hiWhiY hi (14) 

h=1 i∈sh

where Xhi is the mhi × p matrix of covariates for sample units in cluster hi, Whi = diag(wt), t ∈ shi, is the 

diagonal matrix of survey weights for units in cluster hi and Y hi is the mhi × 1 vector of response variables in cluster 

hi. The model variance of βˆ

SW is:

T

V arM(βˆ

SW ) = (X ̃ X ̃ )

−1Gst (15) 

where 

" # X X

H

T

Gst = Xhi

T WhiRhiWhiXhi (X ̃ X ̃ )

−1

h=1 i∈sh

" # (16)

X

H

T

= Xh

TWhRhWhXh (X ̃ X ̃ )

−1

h=1

with Rhi = V arM(Y hi), Wh = diag(Whi), and Rh = Blkdiag(Rhi), Wh = diag(Whi), 

XT = (XT

h1

, XT), i ∈ sh. = (XT

1

, X2

T

, ..., XT

h2

, ..., XT Expression (16) is a special case of (9) with XT

H), h h,nh

where Xh is the mh × p matrix of covariates for sample units in stratum h, W = diag(Whi), for h = 1, ..., H and 

i ∈ sh and R = Blkdiag(Rh). 

Based on the development in Scott & Holt (1982, sec. 4), the MEFF matrix Gst can be rewritten for a special case of 

Rh in a way that will make the decomposition proportions in (12) more understandable. Consider the special case of 

(13) with 

CovM(ehi) = σ

2

(1 − ρ)Imhi + σ

2

ρ1mhi1

T

mhi

where Imhi is the mhi × mhi identity matrix and 1mhi is a vector of mhi 1’s. In that case, 

X XT

hWhRhWhXh = (1 − ρ)XThW2hXh + ρ mhiXT

BhiWhiXBhi

2

i∈sh

−1 1

T where XBhi = m Xhi. Suppose that the sample is self-weighting so that Whi = . After some hi 1mhi mhi wImhi

simplification, it follows that 

Gst = w[Ip + (M − Ip)ρ]

PH P where Ip is the p × p identity matrix and M = ( mhiXT

BhiXBhi)(XTW X)

−1

. Thus, if the sample h=1 i∈sh

is self-weighting and ρ is very small, then Gst ≈ wIp and V arM(βˆ

SW ) in (15) will be approximately the same 

as the OLS variance. If so, the SWLS variance decomposition proportions will be similar to the OLS proportions. 

T

In regression problems, ρ often is small since it is the correlation of the errors, εhit = Yhit − xhitβ, for different 

units rather than for Y hit’s. This is related to the phenomenon that design effects for regression coefficients are often 

smaller than for means-a fact first noted by Kish & Frankel (1974). In applications where ρ is larger, the variance 

decomposition proportions in (12) will still be useful in identifying collinearity although they will be affected by 

departures of the model errors from independence. 

Denote the cluster-level residuals as a vector, ehi = Y hi − Xhiβˆ

SW . The estimator of (15) that we consider was 

originally derived from design-based considerations. A linearization estimator, appropriate when clusters are selected 

with replacement, is: 

T

varL(βˆ

SW ) = (X ̃ X ̃ )

−1Gˆ L

(17) 

7 
with the estimated misspecification effect as 

" # X X

H

nh T

Gˆ ∗ ∗ ∗ ∗

L = (ˆgij )p×p = (zhi − z ̄h

)(zhi − z ̄h)

T

(X ̃ X ̃ )

−1

, (18) 

nh − 1

h=1 i∈sh

where z

∗ 1 P ∗ ∗ = XT ˆ  ̄ = z and z hiWhiehi with ehi = Y hi − Xhi , and the variance-covariance matrix h nh i∈s hi hi

h i

βSW

R ˆ PH nh T 1 Tcan be estimated by R = Blkdiag(ehie ) − ehe . h=1 nh−1 hi nh h

Expression (17) is used by the Stata and SUDAAN packages, among others. The estimator varL(βˆ

SW ) is consistent 

and approximately design-unbiased under a design where clusters are selected with replacement (Fuller, 2002). The 

estimator in (17) is also an approximately model-unbiased estimator of (15) (see Liao, 2010). Since the estimator 

varL(βˆ

SW ) is also currently available in software packages, we will use it in the empirical work in section 4. 

Using (12) derived in section 2, the variance decomposition proportion matrix Π for varL(βˆ

SW ) can then be written 

as 

−1 Π = (πjk)p×p = Q

T Q ̄

L L (19)

with Q ˆ  ̄

L = (φkj )p×p = (U2D−2

) · (U2

T GL)

T

and QL is the diagonal matrix with the row sums of QL on the main 

diagonal and 0 elsewhere. 

4 Numerical Illustrations 

In this section, we will illustrate the collinearity measures described in section 3 and investigate their behaviors using 

the dietary intake data from 2007-2008 National Health and Nutrition Examination Survey (NHANES). 

4.1 Description of the Data 

The dietary intake data are used to estimate the types and amounts of foods and beverages consumed during the 24-

hour period prior to the interview (midnight to midnight), and to estimate intakes of energy, nutrients, and other food 

components from those foods and beverages. NHANES uses a complex, multistage, probability sampling design; 

oversampling of certain population subgroups is done to increase the reliability and precision of health status indicator 

estimates for these groups. Among the respondents who received the in-person interview in the mobile examination 

center (MEC), around 94% provided complete dietary intakes. The survey weights were constructed by taking MEC 

sample weights and further adjusting for the additional nonresponse and the differential allocation by day of the week 

for the dietary intake data collection. These weights are more variable than the MEC weights. The data set used in 

our study is a subset of 2007-2008 data composed of female respondents aged 26 to 40. Observations with missing 

values in the selected variables are excluded from the sample which finally contains 672 complete respondents. The 

final weights in our sample range from 6,028 to 330,067, with a ratio of 55:1. The U.S. National Center for Health 

Statistics recommends that the design of the sample is approximated by the stratified selection with replacement of 32 

PSUs from 16 strata, with 2 PSUs within each stratum. 

4.2 Study One: Correlated Covariates 

In the first empirical study, a linear regression model of respondent’s body mass index (BMI) was considered. The 

explanatory variables considered included two demographic variables, respondent’s age and race (Black/Non-black), 

four dummy variables for whether the respondent is on a special diet of any kind, on a low-calorie diet, on a low-fat 

diet, and on a low-carbohydrate diet (when he/she is on diet, value equals 1, otherwise 0), and ten daily total nutrition 

intake variables, consisting of total calories (100kcal), protein (100gm), carbohydrate (100gm), sugar (100gm), dietary 

8 
fiber (100gm), alcohol (100gm), total fat (100gm), total saturated fatty acids (100gm), total monounsaturated fatty 

acids (100gm), and total polyunsaturated fatty acids (100gm). The correlation coefficients among these variables are 

displayed in Table 2. Note that the correlations among the daily total nutrition intake variables are often high. For 

example, the correlations of the total fat intakes with total saturated fatty acids, total monounsaturated fatty acids and 

total polyunsaturated fatty acids are 0.85, 0.97 and 0.93. 

Three types of regressions were fitted for the selected sample to demonstrate different diagnostics. More details about 

these three regression types and their diagnostic statistics are displayed in Table 1. 

TYPE1: OLS regression with estimated σ

2

; the diagnostic statistics are obtained using the standard methods reviewed 

in section 2; 

TYPE2: WLS regression with estimated σ

2

and assuming R = W−1; the scaled condition indexes are estimated 

using (6) and the scaled variance decomposition proportions are estimated using (12). With R = W−1

, these are the 

variance decompositions that will be produced by standard software using WLS and specifying the weights to be the 

survey weights; 

TYPE3: SWLS with estimated Rˆ , when σ

2R is unknown; the scaled condition indexes are estimated using (6); the 

scaled variance decomposition proportions are estimated using (12). 

Table 1: Regression Models and their Collinearity Diagnostic Statistics used in this Experimental Study 

Type Regression Weight var(βˆ) var(βˆk) Matrix for Variance Decomposition Proportion πjk

Method matrix Condition 

Wa Indexes b 

TYPE1 

TYPE2 

TYPE3 

OLS 

WLS 

SWLS 

I

W

W

σˆ

2

(XT X)

−1

σˆ

2

(XTWX)

−1

σˆ

2

(XTWX)

−1XTWRWˆ X(XTWX)−1

P h i

Rˆ H nh T 1 T = h=1 nh−1 Blkdiag(ehiehi) − ehe nh

h

u

2

σ

2Σ

p 2kj c 

j=1 μ2

j

u

2

σ

2Σ

p 2kj d 

j=1 μ2

j

p

σ

2Σ

p u2kjΣi=1gˆiku2ij e 

j=1 μ2

j

XT X

XTWX

XTWX

u

2 2

2kj u

/Σ

p 2kj

μ2 j=1 2

j

μj

u

2 2

2kj u

/Σ

p 2kj

μ2 j=1 2

j

μ

p

j

p u2kjΣi=1gˆiku2ij ˆ

/Σ

p u2kjΣi=1giku2ij

μ2 j=1 2

j

μj

a

In all the regression models, the parameters are estimated by: βˆ = (XT W X)−1XT W Y . 

bThe eigenvalues of this matrix will be used to compute the Condition Indexes for the corresponding regression model. 

cThe terms u2kj and μj are from the singular value decomposition of the data matrix X. 

dThe terms u2kj and μj are from the singular value decomposition of the weighted data matrix X ̃ = W1/2X. 

eThe terms u2kj and μj are from the singular value decomposition (SVD) of the weighted data matrix X ̃ . The term gˆik is the unit 

element of misspecification effect matrix Gˆ. 

Their diagnostic statistics, including the scaled condition indexes and variance decomposition proportions are reported 

in Tables 3, 4 and 5, respectively. To make the tables more readable, only the proportions that are larger than 0.3 are 

shown. Proportions that are less than 0.3 are shown as dots. Note that some terms in decomposition (12) can be 

negative. This leads to the possibility of some "proportions" being greater than 1. This occurs in five cases in Table 

5. Belsley et al. (1980) suggest that a condition index of 10 signals that collinearity has a moderate effect on standard 

errors; an index of 100 would indicate a serious effect. In this study, we consider a scaled condition index greater than 

10 to be relatively large, and ones greater than 30 as large and remarkable. Furthermore, the large scaled variancedecomposition proportions (greater than 0.3) associated with each large scaled condition index will be used to identify 

those variates that are involved in a near dependency. 

In Tables 3, 4 and 5, the weighted regression methods, WLS and SWLS, used the survey-weighted data matrix X ̃

to obtain the condition indexes while the unweighted regression method, OLS, used the data matrix X. The largest 

scaled condition index in WLS and SWLS is 566, which is slightly smaller than the one in OLS, 581. Both of these 

values are much larger than 30 and, thus, signal a severe near-dependency among the predictors in all three regression 

models. Such large condition numbers imply that the inverse of the design matrix, XTW X, may be numerically 

unstable, i.e., small changes in the x data could make large changes in the elements of the inverse. 

The values of the decomposition proportions for OLS and WLS are very similar and lead to the same predictors being 

identified as potentially collinear. Results for SWLS are somewhat different as sketched below. In OLS and WLS, 

six daily total nutrition intake variables–calorie, protein, carbohydrate, alcohol, dietary fiber and total fat–are involved 

in the dominant near-dependency that is associated with the largest scaled condition index. Four daily fat intake 

variables, total fat, total saturated fatty acids, total monounsaturated fatty acids and total polyunsaturated fatty acids, 

are involved in the secondary near-dependency that is associated with the second largest scaled condition index. A 

9 
moderate near-dependency between intercept and age is also shown in all three tables. The associated scaled condition 

index is equal to 38 in OLS and 37 in WLS and SWLS. However, when SWLS is used, sugar, total saturated fatty acids 

and total polyunsaturated fatty acids also appear to be involved in the dominant near-dependency as shown in Table 

5. While, only three daily fat intake variables, total saturated fatty acids, total monounsaturated fatty acids and total 

polyunsaturated fatty acids, are involved in the secondary near-dependency that is associated with the second largest 

scaled condition index. Thus, when OLS or WLS is used, the impact of near-dependency among sugar, total saturated 

fatty acids, total polyunsaturated fatty acids and the six daily total nutrition intake variables is not as strong as the ones 

in SWLS. If conventional OLS or WLS diagnostics are used for SWLS, this near-dependency might be overlooked. 

Rather than using the scaled condition indexes and variance decomposition method (in Tables 3, 4 and 5), an analyst might attempt to identify collinearities by examining the unweighted correlation coefficient matrix in Table 2. 

Although the correlation coefficient matrix shows that almost all the daily total nutrition intake variables are highly 

or moderately pairwise correlated, it cannot be used to reliably identify the near-dependencies among these variables 

when used in a regression. For example, the correlation coefficient between "on any diet" and "on low-calorie diet" is 

relatively large (0.73). This near dependency is associated with a scaled condition index equal to 11 (larger than 10, 

but less than the cutoff of 30) in OLS and WLS (shown in Table 3 and 4) and is associated with a scaled condition 

index equal to 2 (less than 10) in SWLS (shown in Table 5). The impact of this near dependency appears to be not 

very harmful not matter which regression method is used. On the other hand, alcohol is weakly correlated with all the 

daily total nutrition intake variables but is highly involved in the dominant near-dependency shown in the last row of 

Tables 3-5. 

10 
total.fat 

1 

0.85 0.97 0.93 

alcohol 

1 . 

. . . 

fiber 

1 .0.48 

0.46 0.46 0.43 

sugar 

1 ... 

. . . 

ydrate 

X

carboh

1 

0.84 0.54 

matrix

. 0.54 

0.47 0.51 0.51 

protein 1 

0.45 

data

. 

0.52 

.

0.72 

0.56 0.68 0.71 

0.75 

0.58 

0.86 

0.74 0.83 0.81 

the

calorie 1 

0.84 

0.57 .

Matrix of

low-carb 

a 

on diet 

ficient

1 .. 

. 

. 

. . . 

. . . 

w-fat 

Coef

lo

Correlation

on diet 1 . . . 

.

.

.. . 

...

low-calorie 

2:

on diet 

able

1 . . . . 

. 

. 

. . . 

. . . 

T

on any diet 

c 

1 0.87. . . . 

.

.

.. . 

...

ydrate.

black 

ger than 0.5 is highlighted in this table.

1 . . . . ..

.

.

... 

...

age 

Fatty Acids

Fatty Acids 

b 

1. . . . . ..

.

.

...

...

ficient less than 0.3 is omitted in this table.ficient lar

Fatty Acids

diet 

diet 

w-calorie diet 

at diet 

w-carb

e 

w-f

at

f 

calorie 

at

ar 

alcohol 

d 

y

at 

black on lo

lo

an

protein 

carb 

fiber total.f

at

lo

otal Saturated

mono.f

.f

age onpoly

a The term “carb" stands for carbohb The correlation coef

c The correlation coef

TTotal MonounsaturatedTotal Polyunsaturated

d 

onone

sug

sat.f

e 

f 

11 
w-carb Diet 

c at

on LoMono.f

OLS

. 

. 0.574 . . . . . . . . .... . . .............. .0.890 . 

TYPE1: 

w-fat Diet 

b 

using

Lo

at

on

. . .. 0.379 . . . . . . .... . . 

Sat.f

.............. .0.866 . 

ariance Decomposition Proportions: 

of

Diet 

the Variance

w-Calorie

at 

of

F

Proportion

on Lo

. . . . 0.820 

otal

. . .. . . . . . . . . 

T

. . . . . . . . . . . . . . . 0.304 0.696 

Scaled

yan Diet 

V

on

es and

.

.. .. .....0.842.... .. 

Alcohol 

................ 0.986 

Scaled Condition Index

Black 

.

....0.794...........

DietaryFiber 

................ 0.482 

ar 

Age ..............0.960..Sug.............0.633.. . 

ydrate 

Intercept 

able 3: 

a 

... .. ...... ...0.970.. Carboh................ 0.988 

T

ariance decomposition proportions smaller than 0.3 is omitted in this table.

Fatty Acids

x 

Fatty Acids 

Inde

Fatty Acids

Index 

Condition

The scaled v

otal Saturated

Scaled Condition

TTotal MonounsaturatedTotal Polyunsaturated

a b 

c 

d 

1 2 3 3 3 4 5 6 8 9 11 12 22 26 38 157 581 Scaled1 2 3 3 3 4 5 6 8 9 11 12 22 26 38 157 581 

12 
w-carb Diet 

c at

on LoMono.f

WLS

. 

. 0.609 . . . . . . .. .... . . .............. .0.909 . 

TYPE2:

w-fat Diet 

b 

using

Lo

at

on

. . .. 0.347 . . . . .. .... . . 

Sat.f

.............. .0.871 . 

ariance Decomposition Proportions: 

of

Diet 

the Variance

w-Calorie

at 

of

F

Proportion

on Lo

. . . . 0.878 

otal

. . .. . . . . . . . . 

T

. . . . . . . . . . . . . . . 0.342 0.658 

Scaled

yan Diet 

V

on

es and

.

.. .. .....0.902.... .. 

Alcohol 

................ 0.981 

Scaled Condition Index

Black 

.

....0.711...........

DietaryFiber 

................ 0.486 

ar 

Age ..............0.940..Sug.............0.630.. . 

ydrate 

Intercept 

able 4: 

a 

... .. ...... ...0.959.. Carboh................ 0.987 

T

ariance decomposition proportions smaller than 0.3 is omitted in this table.

Fatty Acids

x 

Fatty Acids 

Inde

Fatty Acids

Index 

Condition

The scaled v

otal Saturated

Scaled Condition

TTotal MonounsaturatedTotal Polyunsaturated

a b 

c 

d 

1 2 3 3 3 4 5 7 8 10 11 13 21 26 37 165 566 Scaled1 2 3 3 3 4 5 7 8 10 11 13 21 26 37 165 566 

13 
Table 5: Scaled Condition Indexes and Variance Decomposition Proportions: using TYPE3: SWLS

Scaled Scaled Proportion of the Variance of

Condition Index Intercept Age Black on any Diet on Low-Calorie Diet on Low-fat Diet on Low-carb Diet 

a 1 . . . . . . . 

2 . . . 0.717 1.278 0.553 . 3 . . . . . . 0.697 3 . . . . . . . 3 . . . . . . . 4 . . . . . . . 5 . . . . . . . 7 0.766 1.686 0.461 . . . . 8 . . . . . . . 10 . . . . . . . 11 . . . . . . . 13 . . . . . . . 21 . . . . . . . 26 . . . . . . . 37 . . . . . . . 165 . . . . . . . 566 0.318 . . . . . . Scaled Condition Index Carbohydrate Sugar Dietary Alcohol Total Fat Sat.fatb Mono.fatc 

Fiber 

1 . . . . . . . 2 . . . . . . . 3 . . . . . . . 3 . . . . . . . 3 . . . . . . . 4 . . . . . . . 5 . . . . . . . 7 . . . . . . . 8 . . . . . . . 10 . . . . . . . 11 . . . . . . . 13 . . . . . . . 21 . . . . . . . 26 . 0.379 . . . . . 37 . . . . . . . 165 . . . . . 0.651 0.749 566 1.008 1.509 0.740 1.036 0.805 0.486 . 

a The scaled variance decomposition proportions smaller than 0.3 is omitted in this table.b Total Saturated Fatty Acids

c Total Monounsaturated Fatty Acids

d Total Polyunsaturated Fatty Acids 

14 
After the collinearity patterns are diagnosed, the common corrective action would be to drop the correlated variables, 

refit the model and reexamine standard errors, collinearity measures and other diagnostics. Omitting X’s one at a 

time may be advisable because of the potentially complex interplay of explanatory variables. In this example, if the 

total fat intake is one of the key variables that an analyst feels must be kept, sugar might be dropped first followed by 

protein, calorie, alcohol, carbohydrate, total fat, dietary fiber, total monounsaturated fatty acids, total polyunsaturated 

fatty acids and monounsaturated fatty acids. Other remedies for collinearity could be to transform the data or use some 

specialized techniques such as ridge regression and mixed Bayesian modeling, which require extra (prior) information 

beyond the scope of most research and evaluations. 

To demonstrate how the collinearity diagnostics can improve the regression results in this example, Table 6 presents 

the SWLS regression analysis output of the original models with all the explanatory variables and a reduced model 

with fewer explanatory variables. In the reduced model, all of the dietary intake variables are eliminated except total 

fat intake. After the number of correlated offending variables is reduced, the standard error of total fat intake is only the 

one forty-sixth of its standard error in the original model. The total fat intake becomes significant in the reduced model. 

The reduction of correlated variables appears to have substantially improved the accuracy of estimating the impact of 

total fat intake on BMI. Note that the collinearity diagnostics do not provide a unique path toward a final model. 

Different analysts may make different choices about whether particular predictors should be dropped or retained. 

Table 6: Regression Analysis Output using TYPE3: SWLS 

Original Model Reduced Model 

Variable Coefficient SEa Coefficient SE 

Intercept 24.14***b 2.77 24.20*** 2.69 

Age 0.06 0.08 0.06 0.08 

Black 3.19*** 1.04 3.67*** 0.98 

on any Dietc 1.79 1.52 1.28 1.80 

on Low-calorie Diet 4.09** 1.50 4.59** 1.69 

on Low-fat Diet 3.67 2.86 3.87 3.76 

on Low-carb Diet 0.46 3.51 0.87 3.86 

Calorie -0.88 2.36 

Protein 7.05 9.59 

Carbohydrate 3.69 9.62 

Sugar -0.31 1.11 

Dietary Fiber -14.52* 5.89 

Alcohol 2.09 16.47 

Total Fat 29.34 31.37 1.47* 0.68 

Total Saturated Fatty Acids -15.90 20.18 

Total Monounsaturated Fatty Acids -22.40 23.01 

Total Polyunsaturated Fatty Acids -27.69 21.10 

Intracluster Coefficient ρ 0.0366 0.0396 

a

standard error 

bp-value: *, 0.05; **, 0.01; ***, 0.005 

cThe reference category is "not being on diet" for all the on-diet variables here. 

4.3 Study Two: Reference Level for Categorical Variables 

As noted earlier, using non-survey data, dummy variables can also play an important role as a possible source for 

collinearity. The choice of reference level for a categorical variable may affect the degree of collinearity in the data. 

To be more specific, choosing a category that has a low frequency as the reference and omitting that level in order to fit 

the model may give rise to collinearity with the intercept term. This phenomenon carries over to survey data analysis 

as we now illustrate. 

We employed the four on-diet dummy variables used in the previous study, which we denote this section as “on 

any diet" (DIET), “on low-calorie diet" (CALDIET), “on low-fat diet" (FATDIET) and “one low-carbohydrate diet" 

15 
(CARBDIET). The model considered here is: 

BMIhit = β0 + βblack ∗ blackhit + βTOTAL.FAT ∗ TOTAL.FAThit + βDIET ∗ DIEThit+

βCALDIET ∗ CALDIEThit + βFATDIET ∗ FATDIEThit+ (20) 

βCARBDIET ∗ CARBDIEThit + εhit

where subscript hit stands for the tth unit in the selected PSU hi, black is the dummy variable of black (black=1 and 

non-black=0), and TOTAL.FAT is the variable of daily total fat intake. According to the survey-weighted frequency 

table, 15.04% of the respondents are “on any diet", 11.43% of them are “on low-calorie diet", 1.33% of them are 

“on low-fat diet" and 0.47% of them are “on low-carbohydrate diet". Being on a diet is, then, relatively rare in this 

example. If we choose the majority level, “not being on the diet", as the reference category for all the four on-diet 

dummy variables, we expect no severe collinearity between dummy variables and the intercept, because most of values 

in the dummy variables will be zero. However, when fitting model (20), assume that an analyst is interested to see 

the impact of “not on any diet" on respondent’s BMI and reverses the reference level of variable DIET in model 

(20) into “being on the diet". This change may cause a near dependency in the model because the column in X for 

variable DIET will nearly equal the column of ones for the intercept. The following empirical study will illustrate the 

impact of this change on the regression coefficient estimation and how we should diagnose the severity of the resulting 

collinearity. 

Table 7 and 8 present the regression analysis output of the model in (20) using the three regression types, OLS, WLS 

and SWLS, listed in Table 1. Table 7 is modeling the effects of on-diet factors on BMI by treating “not being on the 

diet" as the reference category for all the four on-diet variables. While Table 8 changes the reference level of variable 

DIET from “not on any diet" into “On any diet" and models the effect of “not on any diet" on BMI. The choice of 

reference level effects the sign of the estimated coefficient for variable DIET but not its absolute value or standard 

error. The size of the estimated intercept and its SE are different in Tables 7 and 8, but the estimable functions, like 

predictions, will of course, be the same with either set of reference levels. The SE of the intercept is about three times 

larger when “on any diet" is the reference level for variable DIET (Table 8) than when it is not (Table 7). 

Table 7: Regression Analysis Output: When “not on any diet" is the Reference Category for DIET variable 

in the Model 

Regression Intercept black total.fat on any diet on low-calorie diet on low-fat on low-carb diet 

Type diet 

TYPE1 27.22***a 3.20*** 0.95 3.03 1.75 2.75 -1.48 

OLS (0.61)b (0.70) (0.72) (1.94) (2.03) (2.72) (3.66) 

TYPE2 26.13*** 3.65*** 1.44* 1.39 4.46* 3.86 0.94 

WLS (0.58) (0.82) (0.67) (1.67) (1.79) (2.59) (4.22) 

TYPE3 26.13*** 3.65*** 1.44* 1.39 4.46** 3.86 0.94 

SWLS (0.64) (0.99) (0.63) (1.80) (1.70) (3.73) (3.87) 

ap-value: *, 0.05; **, 0.01; ***, 0.005 

bStandard errors are in parentheses under parameter estimates. 

Table 8: Regression Analysis Output: When “on any diet" is the Reference Category for DIET variable in 

the Model 

Regression Intercept black total.fat not on any on low-calorie diet on low-fat on low-carb diet 

Type diet diet 

TYPE1 30.25***a 3.20*** 0.95 -3.03 1.75 2.75 -1.48 

OLS (2.00)b (0.70) (0.72) (1.94) (2.03) (2.72) (3.66) 

TYPE2 27.52*** 3.65*** 1.44* -1.39 4.46* 3.86 0.94 

WLS (1.71) (0.82) (0.67) (1.67) (1.79) (2.59) (4.22) 

TYPE3 27.52*** 3.65*** 1.44* -1.39 4.46** 3.86 0.94 

SWLS (1.75) (0.99) (0.63) (1.80) (1.70) (3.73) (3.87) 

ap-value: *, 0.05; **, 0.01; ***, 0.005 

bStandard errors are in parentheses under parameter estimates. 

When choosing “not being on diet" as the reference category for all the four on-diet dummy variables in Table 9, the 

scaled condition indexes are relatively small and do not signify any remarkable near-dependency regardless of the 

16 
type of regression. Only the last row for the largest condition index is printed in Tables 9 and 10. Often, the reference 

category for a categorical predictor will be chosen to be analytically meaningful. In this example, using “not being on 

diet" for each of the four diet variables would be logical. 

In Table 10, when “on any diet" is chosen as the reference category for variable DIET, the scaled condition indexes 

are increased and show a moderate degree of collinearity (condition index larger than 10) between the on-diet dummy 

variables and the intercept. Using the table of scaled variance decomposition proportions, in OLS and WLS, dummy 

variable for “not on any diet"" and “on low-calorie diet" are involved in the dominant near-dependency with the intercept; however, in SWLS, only the dummy variable for "not on any diet" is involved in the dominant near-dependency 

with the intercept and the other three on-diet variables are much less worrisome. 

Table 9: Largest Scaled Condition Indexes and Its Associated Variance Decomposition Proportions: When 

“not on any diet" is the Reference Category for variable DIET in the Model 

Scaled Scaled Proportion of the Variance of 

Condition Intercept gender total.fat on any diet on low-calorie diet on low-fat diet on low-carb 

Index diet 

TYPE1: OLS 

6 0.005 0.000 0.016 0.949 0.932 0.157 0.200 

TYPE2: WLS 

6 0.013 0.008 0.020 0.938 0.926 0.189 0.175 

TYPE3: SWLS 

6 0.006 0.007 0.013 0.686 0.741 0.027 0.061 

Table 10: Largest Scaled Condition Indexes and Its Associated Variance Decomposition Proportions: When 

“on any diet" is the Reference Category for variable DIET in the Model 

Scaled Scaled Proportion of the Variance of 

Condition Intercept gender total.fat not on any diet on low-calorie diet on low-fat diet on low-carb 

Index diet 

TYPE1: OLS 

17 0.982 0.001 0.034 0.968 0.831 0.155 0.186 

TYPE2: WLS 

17 0.982 0.011 0.029 0.968 0.820 0.182 0.160 

TYPE3: SWLS 

17 0.897 0.018 -0.006 0.971 0.318 0.014 -0.019 

5 Conclusion 

Dependence between predictors in a linear regression model fitted with survey data affects the properties of parameter 

estimators. The problems are the same as for non-survey data: standard errors of slope estimators can be inflated and 

slope estimates can have illogical signs. In the extreme case when one column of the design matrix is exactly a linear 

combination of others, the estimating equations cannot be solved. The more interesting cases are ones where predictors 

are related but the dependence is not exact. The collinearity diagnostics that are available in standard software routines 

are not entirely appropriate for survey data. Any diagnostic that involves variance estimation needs modification to 

account for sample features like stratification, clustering, and unequal weighting. This paper adapts condition numbers 

and variance decompositions, which can be used to identify cases of less than exact dependence, to be applicable for 

survey analysis. 

A condition number of a survey-weighted design matrix W1/2X is the ratio of the maximum to the minimum eigenvalue of the matrix. The larger the condition number the more nearly singular is XTW X, the matrix which must 

be inverted when fitting a linear model. Large condition numbers are a symptom of some of the numerical problems 

associated with collinearity. The terms in the decomposition also involve "misspecification effects" if the model errors 

are not independent as would be the case in a sample with clustering. The variance of an estimator of a regression 

parameter can also be written as a sum of terms that involve the eigenvalues of W1/2X. The variance decompositions 

for different parameter estimators can be used to identify predictors that are correlated with each other. After identifying which predictors are collinear, an analyst can decide whether the collinearity has serious enough effects on a fitted 

17 
model that action should be taken. The simplest step is to drop one or more predictors, refit the model, and observe 

how estimates change. The tools we provide here allow this to be done in a way appropriate for survey-weighted 

regression models. 

References 

Belsley, D. A. (1984). Demeaning conditioning diagnostics through centering. The American Statistician, 38(2), 

73–77. 

Belsley, D. A. (1991). Conditioning Diagnostics, Collinearity and Weak Data in Regression. New York: John Wiley 

and Sons. 

Belsley, D. A., Kuh, E., & Welsch, R. E. (1980). Regression Diagnostics: Identifying Influential Data and Sources of 

Collinearity. Wiley Series in Probability and Statistics. New York: Wiley Interscience. 

Cook, R. D. (1984). Comment on demeaning conditioning diagnostics through centering. The American Statistician, 

2, 88–90. 

Elliot, M. (2007). Bayesian weight trimming for generalized linear regression models. Survey Methodology, 33, 

23–34. 

Farrar, D. E., & Glauber, R. R. (1967). Multicollinearity in regression analysis. Review of Economics and Statistics, 

49, 92–107. 

Fox, J. (1986). Linear Statistical Models and Related Methods, With Applications to Social Research. New York: John 

Wiley. 

Fuller, W. A. (2002). Regression estimation for survey samples. Survey Methodology, 28(1), 5–23. 

Hendrickx, J. (2010). perturb: Tools for evaluating collinearity. R package version 2.04. 

URL http://CRAN.R-project.org/package=perturb

Kish, L., & Frankel, M. (1974). Inference from complex samples. Journal Of the Royal Statistical Society B, 36(1), 

1–37. 

Li, J. (2007a). Linear regression diagnostics in cluster samples. ASA Proceedings of the Section on Survey Research 

Methods, (pp. 3341–3348). 

Li, J. (2007b). Regression diagnostics for complex survey data. Unpublished doctoral dissertation, University of 

Maryland. 

Li, J., & Valliant, R. (2009). Survey weighted hat matrix and leverages. Survey Methodology, 35(1), 15–24. 

Li, J., & Valliant, R. (2011). Linear regression influence diagnostics for unclustered survey data. Journal of Official 

Statistics, 27. 

Liao, D. (2010). Collinearity Diagnostics for Complex Survey Data. Ph.D. thesis, University of Maryland. 

Liao, D., & Valliant, R. (2010). Variance inflation factors in the analysis of complex survey data. submitted. 

Lin, C. (1984). Extrema of quadratic forms and statistical applications. Communications in Statistics-Theory and 

Methods, 13, 1517 – 1520. 

Marquardt, D. W. (1980). Comment on “a critique on some ridge regression methods" by G. smith and F. compbell: 

“You should standardize the predictor variables in your regression models". Journal of the American Statistical 

Association, 75(369), 87–91. 

Scott, A. J., & Holt, D. (1982). The effect of two-stage sampling on ordinary least squares methods. Journal of the 

American Statistical Association, 77(380), 848–854. 

18 
Silvey, S. D. (1969). Multicollinearity and imprecise estimation. Journal of the Royal Statistical Society, 31(3), 

539–552. 

Snee, R. D., & Marquardt, D. W. (1984). Collinearity diagnostics depend on the domain of prediction, and model, and 

the data. The American Statistician, 2, 88–90. 

Steward, G. W. (1987). Collinearity and least squares regression. Statistical Science, 2(1), 68–84. 

Theil, H. (1971). Principles of Econometrics. New York: John Wiley. 

Wissmann, M., Toutenburg, H., & Shalabh (2007). Role of categorical variables in multicollinearity in the linear 

regression model. Technical Report Number 008, Department of Statistics, University of Munich. Available at 

http://epub.ub.uni-muenchen.de/2081/1/report008_statistics.pdf. 

Wood, F. S. (1984). Effect of centering on collinearity and interpretation of the constant. The American Statistician, 

2, 88–90. 

19