structural modeling and psychometrika an historical perspective on growth and achievements the field of linear structural equation modeling with continuous variables is reviewed trends in psychometric theory and data analysis across the five decades of publication ofpsychometrika are discussed especially the clarification of concepts of population and sample explication of the parametric structure of models delineation of concepts of exploratory and confirmatory data analysis expansion of statistical theory in psychometrics estimation via optimization of an explicit objective function and implementation of general function minimization methods developments in the ideas of factor analysis latent variables as well as structural and causal modeling are noted some major conceptual achievements involving general covariance structure representations multiple population models and moment structures are reviewed the major statistical achievements of normal theory generalized least squares estimation elliptical and distribution free estimation and higher moment estimation are discussed computer programs that implement some of the theoretical developments are described

efficient estimation in the errors in variables model we consider efficient estimation of the slope in the errors in variables model with normal error when either the ratio of error variances is known and the distribution of the independent is arbitrary and unknown or the distribution of the independent variable is not gaussian or degenerate we calculate information bounds and exhibit estimates achieving these bounds using an initial minimum distance estimate and suitable estimates of the efficient score function

generalized m estimators for errors in variables regression this paper discusses robust estimation for structural errors in variables ev linear regression models such models have important applications in many areas under certain assumptions including normality the maximum likelihood estimates for the ev model are provided by orthogonal regression or which minimizes the orthogonal distance from the regression line to the data points instead of the vertical distance used in ordinary regression or is very sensitive to contamination and thus efficient robust procedures are needed this paper examines the theoretical properties of bounded influence estimators for univariate gaussian ev models using a generalized m estimate approach the results include fisher consistency most b robust estimators and the or version of hampel s optimality problem

the limiting distribution of least squares in an errors in variables regression model abstract there is a substantial literature concerning linear regression when some of the predictors independent variables are measured with error such models are of importance in econometrics instrumental variables models psychometrics correction for attenuation models of change and in instrumental calibration studies in medicine and industry recent theoretical work concerning maximum likelihood estimation in such models appears in healy 1980 fuller 1980 and anderson 1984 while reilly and patino leal 1981 take a bayesian approach in an applied context an investigator may either overlook the measurement errors in the predictors or choose the classical ordinary least squares ols estimator of the parameters because of its familiarity and ease of use certainly the methodology of classical least squares theory confidence intervals multiple comparisons tests of hypotheses residual analysis is considerably more developed than the corresponding errors in variables methodology particularly in samples of moderate size in this paper it is shown that under reasonable regularity conditions such linear combinations are jointly asymptotically normally distributed

very strong approximation for certain algebraic varieties let f be a global field in this work we show that the brauer manin condition on adelic points for subvarieties of a torus t f over f cuts out exactly the rational points if either f is a function field or if f mathbb q and t f is split as an application we prove a conjecture of harari voloch over global function fields which states roughly speaking that on any rational hyperbolic curve the local integral points with the brauer manin condition are the global integral points finally we prove for tori over number fields a theorem of stoll on adelic points of zero dimensional subvarieties in abelian varieties

algebraic connections between the least squares and total least squares problems in this paper the closeness of the total least squares tls and the classical least squares ls problem is studied algebraically interesting algebraic connections between their solutions their residuals their corrections applied to data fitting and their approximate subspaces are proven all these relationships point out the parameters which mainly determine the equivalences and differences between the two techniques these parameters also lead to a better understanding of the differences in sensitivity between both approaches with respect to perturbations of the data in particular it is shown how the differences between both approaches increase when the equationsax b become less compatible when the length ofb orx is growing or whena tends to be rank deficient they are maximal whenb is parallel with the singular vector ofa associated with its smallest singular value furthermore it is shown how tls leads to a weighted ls problem and assumptions about the underlying perturbation model of both techniques are deduced it is shown that many perturbation models correspond with the same tls solution

identifiability of factor analysis some results and open problems abstract identifiability of the factor analysis model is discussed and some recent results and their relation to previous work are surveyed sufficient conditions for local identifiability are presented a counterexample shows that these conditions are not necessary in general a simple counting rule for local identifiability of the factor analysis model is given generically finally some open problems are formulated

asymptotic properties of estimators for autoregressive models with errors in variables let x t t z be an observable strictly stationary sequence of random variables and let x t u t e t where u t is an ar p and e t is a strictly stationary sequence representing errors of measurement in x t with e e 1 0 under some broad assumptions on e t we establish the consistency properties as well as the rates of convergence for the standard estimators for the autoregressive parameters computed from a set of modified yule walker equations

on the relations among regular equal unique variances and image factor analysis models we investigate under what conditions the matrix of factor loadings from the factor analysis model with equal unique variances will give a good approximation to the matrix of factor loadings from the regular factor analysis model we show that the two models will give similar matrices of factor loadings if schneeweiss condition that the difference between the largest and the smallest value of unique variances is small relative to the sizes of the column sums of squared factor loadings holds furthermore we generalize our results and discus the conditions under which the matrix of factor loadings from the regular factor analysis model will be well approximated by the matrix of factor loadings from joreskog s image factor analysis model especially we discuss guttman s condition i e the number of variables increases without limit for the two models to agree in relation with the condition we have shown and conclude that schneeweiss condition is a generalization of guttman s condition some implications for practice are discussed

evaluation of the distribution function of the limited information maximum likelihood estimator the distributions of the limited information maximum likelihood estimator for the coefficient of one endogenous variable are evaluated numerically tables are given for enough values of the parameters to cover all cases of interests comparisons are made with the two stage least squares estimator

generic global indentification in factor analysis abstract the paper proves global indentification of the unique variances in factor analysis for almost all parameter points if the number of factors is smaller than the ledermann bound it also shows how to compute using a closed form representation the second solution in the case of six variables and three factors

the prejudices of least squares principal components and common factors schemes we prove mathematically that the least squares regression scheme is of little use for identification from inexact data even in the low noise case its results depend solely on the prejudice concerning which subsets of variables are chosen as regressands and which as regressors from a given set of data variables in practice least squares regression results are always biased and depend on the relative noise levels while even the signs of the estimates are completely determined by the arbitrary choice of the regressands if the number of underlying relationships is misspecified the principal components or statistical common factor scheme fares a similar fate since the choice of how many principal components or common factors to retain is essentially prejudiced and not determined by the data both schemes produce artificial and unexplained correlations among the residuals for the principal components scheme this occurs because it usually violates wilson s inequality a fortiori we prove exactly why in practice factor indeterminancy occurs the exact ideal multiple common factor scheme of frisch and thurstone has not yet been solved except for small numbers six and less of data variables

the nonexistence of 100 1 alpha confidence sets of finite expected diameter in errors in variables and related models on montre que pour certains modeles il est impossible de construire des intervalles de confiance pour les parametres cles qui aient a la fois une confiance positive et une longueur esperee finie

the asymptotic covariance matrix of maximum likelihood estimates in factor analysis the case of nearly singular matrix of estimates of unique variances this paper is concerned with the asymptotic covariance matrix acm of maximum likelihood estimates mles of factor loadings and unique variances when one element of mles of unique variances is nearly zero i e the matrix of mles of unique variances is nearly singular in this situation standard formulas break down we give explicit formulas for the acm of mles of factor loadings and unique variances that could be used even when an element of mles of unique variances is very close to zero we also discuss an alternative approach using the augmented information matrix under a nearly singular matrix of mles of unique variances and derive the partial derivatives of the alternative constraint functions with respect to the elements of factor loadings and unique variances 2000 elsevier science inc all rights reserved ams classification 62h25 62f12

a unifying theorem for linear and total linear least squares it is shown how both linear least squares and total linear least squares estimation schemes are special cases of a rank one modification of the data matrix or the sample covariance matrix for a problem with n unknowns there exist n linear least squares solutions while the total linear least squares solution is generically unique when the signal to noise ratio is sufficiently high the total least squares solution is a nonnegative combination of the least squares solutions

maximum likelihood estimation for array processing in colored noise direction of arrival estimation of multiple sources using a uniform linear array in noise with unknown covariance is considered the noise is modeled as a spatial autoregressive process with unknown parameters both stochastic and deterministic signal models are considered for the random signal case an approximate maximum likelihood estimator of the signal and noise parameters is derived it requires numerical maximization of a compressed likelihood function over the unknown arrival angles analytical expressions for the mles of the signal covariance and the ar parameters are given similar results for the case of deterministic signals are also presented

two stage instrumental variables estimators for the nonlinear errors in variables model abstract estimation for the nonlinear functional errors in variables model is considered under the assumption that instrumental variables are available the approximate bias in the ordinary instrumental variable estimator due to the nonlinearity of the relationship is derived utilizing the nonlinearity of the relationship an estimator of the error covariance matrix is introduced using the error covariance matrix estimator two types of two stage instrumental variable estimators are proposed the two two stage estimators do not have the asymptotic bias due to the nonlinearity a monte carlo experiment also shows the superiority of the two stage estimators over the ordinary estimator

nonlinear eigenvector algorithms for local optimization in multivariate data analysis abstract many multivariate statistical procedures such as principal component canonical correlation correspondence and discriminant analysis are based on the solution of a certain matrix approximation problem one of the most important being the reduced rank approximation its solution hinges on the singular value or the eigendecomposition of a certain matrix therefore numerical techniques to determine eigenvalues and eigenvectors play an important role in these statistical applications furthermore various modifications generalizations or refinements of the classical methods lead to some sort of nonlinear eigenproblem with no immediate unique solution however the nonlinear eigenvector approach has proved to be quite effective in solving these types of constrained optimization problems in diverse fields of multivariate data analysis since a general analysis of the properties and performance of the nonlinear eigenvector algorithm seems to be long overdue this paper collects the common features in the applications mentioned above and gives an overview from a superior perspective it unifies the treatment by contrasting the nonlinear eigenvector algorithm with the well known inverse iteration iteratively reweighted least squares and majorization algorithms extracts the mathematical tools basically from convex analysis and matrix algebra needed for the convergence proofs and discusses convergence properties

instrumental variable estimator for the nonlinear errors in variables model abstract estimation for the nonlinear errors in variables model is considered it isassumed that additional information is available in the form of observations on instrumental variables an estimation procedure is presented for the parameters of the model asymptotic properties of the estimator are investigated

measurement error and ancova functional and structural relationship approaches this article discusses alternative procedures to the standardf test for ancova in case the covariate is measured with error both a functional and a structural relationship approach are described examples of both types of analysis are given for the simple two group design several cases are discussed and special attention is given to issues of model identifiability an approximate statistical test based on the functional relationship approach is described on the basis of monte carlo simulation results it is concluded that this testing procedure is to be preferred to the conventionalf test of the ancova null hypothesis it is shown how the standard null hypothesis may be tested in a structural relationship approach it is concluded that some knowledge of the reliability of the covariate is necessary in order to obtain meaningful results

comparison of least squares and errors in variables regression with special reference to randomized analysis of covariance abstract in an errors in variables regression model the least squares estimate is generally inconsistent for the complete regression parameter but can be consistent for certain linear combinations of this parameter we explore the conjecture that when the least squares estimate is consistent for a linear combination of the regression parameter it will be preferred to an errors in variables estimate at least asymptotically the conjecture is false in general but it is true for some important classes of problems one such problem is a randomized two group analysis of covariance upon which we focus

estimation in multivariate errors in variables models abstract this paper reviews and extends some of the known results in the estimation in errors in variables models treating the structural and the functional cases on a unified basis the generalized least squares method proposed by some previous authors is extended to the case where the error covariance matrix contains an unknown vector parameter this alleviates the difficulty of multiple roots arising from defining estimators as roots to a set of unbiased estimating equations an alternative method is also considered for cases with both known and unknown error covariance matrix the relationship between this method and the usual maximum likelihood and generalized least squares approaches is also investigated and it is shown that in a special case they do not necessarily give identical results in finite samples finally asymptotic results are presented

em algorithms for ml factor analysis the details of em algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models the algorithm is essentially the same for both cases and involves only simple least squares regression operations the largest matrix inversion required is for aq q symmetric matrix whereq is the matrix of factors the example that is used demonstrates that the likelihood for the factor analysis model may have multiple modes that are not simply rotations of each other such behavior should concern users of maximum likelihood factor analysis and certainly should cast doubt on the general utility of second derivatives of the log likelihood as measures of precision of estimation

estimating linear statistical relationships this paper on estimating linear statistical relationships includes three lectures on linear functional and structural relationships factor analysis and simultaneous equations models the emphasis is on relating the several models by a general approach and on the similarity of maximum likelihood estimators under normality in the different models in the first two lectures the observable vector is decomposed into a systematic part and a random error the systematic part satisfies the linear relationships estimators are derived for several cases and some of their properties given estimation of the coefficients of a single equation in a simultaneous equations model is shown to be equivalent to estimation of linear functional relationships

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

a structure theory for linear dynamic errors in variables models we deal with problems connected with the identification of linear dynamic systems in situations when inputs and outputs may be contaminated by noise the case of uncorrelated noise components and the bounded noise case is considered if also the inputs may be contaminated by noise a number of additional complications in identification arise in particular the underlying system is not uniquely determined from the population second moments of the observations a description of classes of observationally equivalent systems is given continuity properties of mappings relating classes of observationally equivalent systems to the spectral densities of the observations are derived and the classes of spectral densities corresponding to a given maximum number of outputs are studied

multiplicative errors in variables models with applications to recent data released by the u s department of energy abstract in an errors in variables model the predicting variables are observed with errors traditionally the errors are assumed to be additive in this article i consider the case in which the error is multiplicative a situation that arises when analyzing some recent data released by the u s department of energy a consistent estimator is provided for regression coefficients by correcting the asymptotic bias of the least squares estimate it is shown that after being normalized by an estimate of its covariance is asymptotically normally distributed this can therefore be used to construct approximate tests and confidence sets for the results are essentially nonparametric

estimation of common linear functional relationships in k data sets summary this paper studies a functional type linear measurement error model in which k data sets of observations are assumed to have in common a set of linear restrictions on their respective unobservable constant vectors each data set consists of replicated measurements of each one of several generally unequal constant vectors with normal error assumed independent for each measurement made the error covariance matrices are unspecified and are equal for measurements within a data set but in general unequal among the data sets this work generalises the results of anderson 1951 1984 who derived maximum likelihood estimators for the constant vectors and error covariance matrix of a single such data set the model recalls the common space analysis described by flury 1987 1988 pp 131 4 but differs by the presence of a fixed effects subspace maximum likelihood estimation of the model parameters requires special numerical methods an example from electroencephalography is used to introduce and illustrate the method

panel models with interactive effects abstract the multiplication of individual specific effects and time specific effects provides a more general formulation than the traditionally used additive form to capture the unobserved heterogeneity in panel data modeling it is also a useful approach for dimension reduction for modeling cross section dependence however and are unobservable we explore the implications for econometric modeling under various formulations of the interactive effects models and suggest a quasi likelihood approach as a common framework to study issues of estimation and statistical inference when regressors are either strictly exogenous or predetermined and under different combinations of the data size of cross sectional dimension n and time series dimensions t we also suggest some computationally simpler estimation methods in light of the quasi likelihood approach monte carlo studies are conducted to highlight the issues involved

dimension estimation in noisy pca with sure and random matrix theory principal component analysis pca is one of the best known methods for dimensionality reduction perhaps the most important problem in using pca is to determine the number of principal components pcs or equivalently choose the rank of the loading matrix many methods have been proposed to deal with this problem but almost all of them fail in the important practical case when the number of observation is comparable to the number of variables i e the realm of random matrix theory rmt in this paper we propose to use stein s unbiased risk estimator sure to estimate with some assistance from rmt the number of principal components the method is applied both on simulated and real functional magnetic resonance imaging fmri data and compared to bic and the laplace method

frequency domain system identification using non parametric noise models estimated from a small number of data sets abstract this paper discusses the problem of identifying a linear system from the frequency data when the measurements of the input and the output signals are both disturbed with noise a typical example of such a problem is the identification of a system in a feedback loop it is known that this problem can be solved using errors in varaibles methods if the covariance matrices of the disturbing noise on input and output measurements are a priori known it is shown that the exact covariance matrices can be replaced by the sample covariance matrices the system can be identified from the sample means and sample covariance matrices calculated from a small number m of independently repeated experiments it is shown that under these conditions the estimates are still strongly consistent for an increasing number of data points n in each experiment n if m 4 the loss in efficiency is quantified m 6 and the expected value of the cost function m 4 and its variance m 6 are calculated

nonparametric regression with errors in variables the effect of errors in variables in nonparametric regression estimation is examined to account for errors in covariates deconvolution is involved in the construction of a new class of kernel estimators it is shown that optimal local and global rates of convergence of these kernel estimators can be characterized by the tail behavior of the characteristic function of the error distribution in fact there are two types of rates of convergence according to whether the error is ordinary smooth or super smooth it is also shown that these results hold uniformly over a class of joint distributions of the response and the covariate which is rich enough for many practical applications furthermore to achieve optimality we show that the convergence rates of all possible estimators have a lower bound possessed by the kernel estimators

efficient maximum likelihood estimation in semiparametric mixture models we consider maximum likelihood estimation in several examples of semiparametric mixture models including the exponential frailty model and the errors in variables model the observations consist of a sample of size n from the mixture density int p theta x z d eta z the mixing distribution is completely unknown we show that the first component hat theta n of the joint maximum likelihood estimator hat theta n hat eta n is asymptotically normal and asymptotically efficient in the semiparametric sense

measurement methods comparisions and linear statistical relationship the linear statistical relationship model can be used to compare two measurement procedures crude methods are now typically used in practice because the statistical concepts are often not understood by practitioners to clarify concepts we define three types of equivalency based on accuracy and precision of the measurement methods and show that the confidence interval for slope depends on sample size and information size these insights lead to obtaining bounded confidence intervals and equivalence tests the derived results provide the basis for designing and analyzing a comparison experiment an example from practice is used to illustrate the methodology

consistent estimation for some nonlinear errors in variables models abstract the issues of identification and estimation of nonlinear errors in variables models are explored the deficiencies of the conventional definition of identification are discussed and an alternative definition in terms of observed samples is suggested the consistency and asymptotic normality of minimum distance estimators are studied to simplify the computation a two step estimation procedure is also suggested in which a consistent estimate of a subset of parameters is first obtained and treated as if they were known in the second step estimation of the rest of the parameters conditions for the consistency of this two step estimator and its asymptotic variance covariance matrix are also derived

linear models based on noisy data and the frisch scheme we address the problem of identifying linear relations among variables based on noisy measurements this is a central question in the search for structure in large data sets often a key assumption is that measurement errors in each variable are independent this basic formulation has its roots in the work of charles spearman in 1904 and of ragnar frisch in the 1930s various topics such as errors in variables factor analysis and instrumental variables all refer to alternative viewpoints on this problem and on ways to account for the anticipated way that noise enters the data in the present paper we begin by describing certain fundamental contributions by the founders of the field and provide alternative modern proofs to certain key results we then go on to consider a modern viewpoint and novel numerical techniques to the problem the central theme is expressed by the frisch kalman dictum which calls for identifying a noise contribution that allows a maximal number of simultaneous linear relations a

the lp minkowski problem for n p 1 abstract chou and wang s existence result for the l p minkowski problem on s n 1 for p n 1 and an absolutely continuous measure is discussed and extended to more general measures in particular we provide an almost optimal sufficient condition for the case p 0 1

estimating and testing interactions in linear regression models when explanatory variables are subject to classical measurement error estimating and testing interactions in a linear regression model when normally distributed explanatory variables are subject to classical measurement error is complex since the interaction term is a product of two variables and involves errors of more complex structure our aim is to develop simple methods based on the method of moments mm and regression calibration rc that yield consistent estimators of the regression coefficients and their standard errors when the model includes one or more interactions in contrast to previous work using structural equations models framework our methods allow errors that are correlated with each other and can deal with measurements of relatively low reliability using simulations we show that under the normality assumptions the rc method yields estimators with negligible bias and is superior to mm in both bias and variance we also show that the rc method also yields the correct type i error rate of the test of the interaction however when the true covariates are not normally distributed we recommend using mm we provide an example relating homocysteine to serum folate and b12 levels

on the asymptotic optimality of the liml estimator with possibly many instruments we consider the estimation of the coefficients of a linear structural equation in a simultaneous equation system when there are many instrumental variables we derive some asymptotic properties of the limited information maximum likelihood liml estimator when the number of instruments is large some of these results are new as well as old and we relate them to results in some recent studies we have found that the variance of the limiting distribution of the liml estimator and its modifications often attain the asymptotic lower bound when the number of instruments is large and the disturbance terms are not necessarily normally distributed that is for the micro econometric models of some cases recently called many instruments and many weak instruments