The Robustness of Normal-theory LISREL Models : Tests Using aNew Optimizer , the Bootstrap , and Sampling Experiments

The Robustness of Normal-theory LISREL Models: Tests Using a New Optimizer, the Bootstrap, and Sampling Experiments, with Applications Asymptotic results from theoretical statistics show that the linear structural relations (LISREL) covariance structure model is robust to many kinds of departures from multivariate normality in the observed data. But close examination of the statistical theory suggests that the kinds of hypotheses about alternative models that are most often of interest in political science research are not covered by the nice robustness results. The typical size of political science data samples also raises questions about the applicability of the asymptotic normal theory. We present results from a Monte Carlo sampling experiment and from analysis of two real data sets both to illustrate the robustness results and to demonstrate how it is unwise to rely on them in substantive political science research. We propose new methods using the bootstrap to assess more accurately the distributions of parameter estimates and test statistics for the LISREL model. To implement the bootstrap we use optimization software two of us have developed, incorporating the quasi-Newton BFGS method in an evolutionary programming algorithm. We describe methods for drawing inferences about LISREL models that are much more reliable than the asymptotic normal-theory techniques. The methods we propose are implemented using the new software we have developed. Our bootstrap and optimization methods allow model assessment and model selection to use well understood statistical principles such as classical hypothesis testing. \Distortions. Interference. Real data is messy. It's all very, very noisy out there. Very hard to spot the tune. Like a piano in the next room, it's playing your song, but unfortunately it's out of whack, some of the strings are missing, and the pianist is tone deaf and drunk|I mean, the noise! Impossible! " |Valentine speaking in Tom Stoppard's Arcadia, I:4. Introduction The Linear Structural Relations or LISREL model for covariance structures is often used in political science and related social science elds to specify and estimate stochastic models for linear relations in the presence of measurement errors. Asymptotic results from theoretical statistics show that the model is highly robust to many kinds of departures from multivariate normality in the observed data. But close examination of the statistical theory suggests that the kinds of hypotheses about alternative models that are most often of interest in political science research are not covered by the nice robustness results. The typical size of political science data samples also raises questions about the applicability of the asymptotic normal theory. We present results from a Monte Carlo sampling experiment and from analysis of two real data sets both to illustrate the robustness results and to demonstrate how it is unwise to rely on them in substantive political science research. We propose new methods using the bootstrap to assess more accurately the distributions of parameter estimates and test statistics for the LISREL model. To implement the bootstrap we use optimization software two of us have developed, incorporating a quasi-Newton method in an evolutionary programming algorithm. We describe methods for drawing inferences about LISREL models that are much more reliable than the asymptotic normal-theory techniques. The methods we propose are implemented using the new software we have developed. 1 Bollen (1989) gives an introduction to LISREL models, including good review of theoretical and applied literature up through the mid-1980s. In its development during the 1960s and 1970s, statistical theory for the LISREL model was asymptotic and based on likelihood estimators derived under an assumption of multivariate normality for both the observed and the unobserved data (e.g., J oreskog 1973). But early work on the factor analysis model had previously demonstrated that using a likelihood function based on the assumption of multivariate normality to obtain maximum likelihood estimates gives consistent and asymptotically normal estimates for factor loadings under relatively mild conditions that do not require that the common factors be multivariate normal (Anderson and Rubin 1956). During the 1980s several writers extended that early theory to show similar results for quite general forms of the LISREL model (Amemiya, Fuller and Pantula 1987; Anderson and Amemiya 1988; Browne 1987; Browne and Shapiro 1988; Anderson 1989). Theory now demonstrates that estimation in the general LISREL model using the multivariate normalbased likelihood can give consistent and asymptotically normal parameter estimates even when neither exogenous structural variables, structual equation disturbances nor measurement errors are normally distributed. Indeed, the theory shows that likelihood ratio test statistics have the usual asymptotic 2 distributions in such cases, so that assessments of goodness-oft and tests of hypotheses framed in terms of linear restrictions can proceed in the usual way (Amemiya and Anderson 1990). Procedures based on the normal-theory analysis are inadequate only if the model is misspeci ed, which means that it does not asymptotically converge to the true covariance matrix, or if the hypotheses to be tested involve restrictions on covariance parameters for latent variables that have asymptotically nonzero fourth cumulants (Browne and Shapiro 1988; Anderson 1989). Political science data are rarely if ever multivariate normal, so unquestionably evidence is needed to show that LISREL models that may be applied to them are robust. Very often data are categorical|opinion sample survey response data, for instance, or coded data that implement the 2 categories of a theoretically motivated typology. Or data are amounts (e.g. of money), proportions (e.g. of votes) or counts of events, all kinds of data that must be positive, bounded between zero and one or otherwise restricted. LISREL models have been used in political science research with all these kinds of data. It would therefore be reassuring to have statistical theory that shows one can trust the familiar statistics for parameter estimates, standard errors for those estimates and overall goodness of t. Unfortunately, the remarkable asymptotic robustness results by Anderson and Amemiya and the others do not imply that normal-theory asymptotic analysis is a su cient foundation for drawing reliable inferences using the LISREL model. Often in political science applications, the hypotheses of interest involve alternative claims about the dimensionality of the space of common factors. Or there are competing claims that amount to alternative restrictions on the pattern of covariances among the common factors. Examples of the latter include situations where the covariance matrix of the structural equation disturbances is constrained to be diagonal while attention focuses on alternative restrictions on the structural coe cients that relate the endogenous latent variables to one another. A crucial nuance in the statistical theory is that the asymptotic robustness results do not apply if the latent variables to which such hypotheses are being applied are not su ciently normal-like. To be a bit more precise, for virtually all kinds of data the theory shows that parameter estimates are asymptotically normal, subject to identi ability requirements. But the asymptotic distribution has the same covariance matrix as occurs in the fully normal case only under special circumstances. The details are a bit intricate, but a reasonable summary is to say that the normal-theory results do not apply if (as stated above) the latent variables whose variances or covariances are a ected by constraints have fourth cumulants that do not vanish as the sample size tends to in nity (Browne and Shapiro 1988; Browne 1984). Perhaps more seriously for practice in deciding among alternative 3 speci cations, such constraints also mean that the usual likelihood-ratio and goodness-oft test statistics do not have a limiting central 2 distribution (Anderson 1989, 107). This means that when the latent dimensionality of the data is in question or zero constraints beyond those needed for identi cation have been imposed on parameters that represent relations between endogenous latent variables, it is appropriate to use a reference central 2 distribution only if there is no reasonable doubt about the condition on fourth cumulants. When the observed data are non-normal in the ways that data used in political science applications usually are non-normal, it seems unlikely that such a special condition will be satis ed on a routine basis. Even when the form of the model is such that the robustness results should apply, the size of the available sample may not be large enough to make the asymptotic normal approximations trustworthy. Assuming nite fourth central moments, Portnoy (1988) shows that for common models the normal approximation for the distribution of maximum likelihood estimates (MLEs) for model parameters may not be trustworthy unless q2=n is small, where q is the number of parameters and n is the sample size. Portnoy further shows that the normal (or 2) approximation for a likelihood ratio test of a simple hypothesis may not be trustworthy unless q3=2=n is small. In LISREL models the number of parameters squared can easily become large even in what one would practically think of as a simple speci cation. For instance, to refer to a model form used in one of the examples with real data analyzed below, a con rmatory factor analysis model with one factor and six observed variables has q = 12 free parameters: six measurement error variances; ve factor loadings; and one latent factor variance. Even with completely normal data, a correct speci cation and a sample of size n = 500, studentized quantities for individual parameters will by Po