limited and full information estimation and goodness of fit testing in 2n contingency tables high dimensional contingency tables tend to be sparse and standard goodness of fit statistics such as x2 cannot be used without pooling categories as an improvement on arbitrary pooling for goodness of fit of large 2n contingency tables we propose classes of quadratic form statistics based on the residuals of margins or multivariate moments up to order r these classes of test statistics are asymptotically chi squared distributed under the null hypothesis further the marginal residuals are useful for diagnosing lack of fit of parametric models we show that when r is small r 2 3 the proposed statistics have better small sample properties and are asymptotically more powerful than x2 for some useful multivariate binary models related to these test statistics is a class of limited information estimators based on low dimensional margins we show that these estimators have high efficiency for one commonly used latent trait model for binary data

characterizing the manifest probabilities of latent trait models the problem of characterizing the manifest probabilities of a latent trait model is considered the item characteristic curve is transformed to the item passing odds curve and a corresponding transformation is made on the distribution of ability this results in a useful expression for the manifest probabilities of any latent trait model the result is then applied to give a characterization of the rasch model as a log linear model for a 2 j contingency table partial results are also obtained for other models the question of the identifiability of guessing parameters is also discussed

quasiregularity and rigorous diffusion of strong hamiltonian chaos exact results are derived concerning quasiregularity and diffusion of strong chaos on resonances of the sawtooth map a chaotic ensemble of well defined quasiregularity type the sequence of resonances visited is generally a fractal set whose main characteristics the topological entropy and the hausdorff dimension are calculated exactly under some conditions using a symbolic dynamics the effect of quasiregularity on chaotic diffusion is characterized by an infinity of diffusion coefficients each associated with a fractal ensemble trapped in a periodic set of resonances in some cases these coefficients are calculated exactly and it is shown that rigorous diffusion takes place on the resonances

on two functions of a matrix with positive definite hermitian part let a be a square complex matrix with positive definite hermitian part h a a a h 2 and let f a g a be two functions defined by f a h a 1 1 g a a 1 a h this paper derives

a nonparametric approach for assessing latent trait unidimensionality assuming a nonparametric family of item response theory models a theory based procedure for testing the hypothesis of unidimensionality of the latent space is proposed the asymptotic distribution of the test statistic is derived assuming unidimensionality thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait based upon a new notion of dimensionality the test is shown to have asymptotic power 1 a 6300 trial monte carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice

fast iterative kernel principal component analysis we develop gain adaptation methods that improve convergence of the kernel hebbian algorithm kha for iterative kernel pca kim et al 2005 kha has a scalar gain parameter which is either held constant or decreased according to a predetermined annealing schedule leading to slow convergence we accelerate it by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector an additional normalization term then allows us to eliminate a tuning parameter in the annealing schedule finally we derive and apply stochastic meta descent smd gain vector adaptation schraudolph 1999 2002 in reproducing kernel hilbert space to further speed up convergence experimental results on kernel pca and spectral clustering of usps digits motion capture and image denoising and image super resolution tasks confirm that our methods converge substantially faster than conventional kha to demonstrate scalability we perform kernel pca on the entire mnist data set

goodness of fit tests for log linear models in sparse contingency tables abstract the asymptotic normality of the likelihood ratio goodness of fit statistic is demonstrated for testing the fit of log linear models with closed form maximum likelihood estimates in sparse contingency tables unlike the traditional chi squared theory the number of categories in the table increases as the sample size increases but not all of the expected frequencies are required to become large some results of a small monte carlo study are presented the traditional chi squared approximation is reasonably accurate for the pearson statistic for many sparse tables but cases are presented for which it fails the normal approximation can be much more accurate than the chi squared approximation for the likelihood ratio statistic but the bias of estimated moments is a potential problem for very sparse tables

the generalized logit linear item response model for binary designed items this paper introduces the generalized logit linear item response model gllirm which represents the item solving process as a series of dichotomous operations or steps the gllirm assumes that the probability function of the item response is a logistic function of a linear composite of basic parameters which describe the operations and the coefficients depend on three design matrices x y and z the gllirm provides a tool for testing hypotheses on the item solving process and generalizes existing models an empirical application is included in which the model is applied to evaluate sources of difficulty and pairwise item interactions in a logical analysis test

analysis of residuals for the multinomial item response model using the item response model as developed on the multinomial distribution asymptotic variances are obtained for residuals associated with response patterns and first and second order marginal frequencies of manifest variables when the model does not fit well an examination of these residuals may reveal the source of the poor fit finally a limited information test of fit for the model is developed by using residuals defined for the first and second order marginals model evaluation based on residuals for these marginals is particularly useful when the response pattern frequencies are sparse

a limited feedback joint precoding for amplify and forward relaying this paper deals with the practical precoding design for a dual hop downlink with multiple input multiple output mimo amplify and forward relaying first assuming that full channel state information csi of the two hop channels is available a suboptimal dual hop joint precoding scheme i e precoding at both the base station and relay station is investigated based on its structure a scheme of limited feedback joint precoding using joint codebooks is then proposed which uses a distributed codeword selection to concurrently choose two joint precoders such that the feedback delay is considerably decreased finally the joint codebook design for the limited feedback joint precoding system is analyzed and results reveal that independent codebook designs at the base station and relay station using the conventional grassmannian subspace packing method is able to guarantee that the overall performance of the dual hop joint precoding scheme improves with the size of each of the two codebooks simulation results show that the proposed dual hop joint precoding system using distributed codeword selection scheme exhibits a rate or ber performance close to the one using the optimal centralized codeword selection scheme while having lower computational complexity and shorter feedback delay

general results on the convergence of stochastic algorithms a deterministic approach is proposed for proving the convergence of stochastic algorithms of the most general form under necessary conditions on the input noise and reasonable conditions on the nonnecessarily continuous mean field emphasis is placed on the case where more than one stationary point exists we also use this approach to prove the convergence of a stochastic algorithm with markovian dynamics

mcmc estimation and some model fit analysis of multidimensional irt models a bayesian procedure to estimate the three parameter normal ogive model and a generalization of the procedure to a model with multidimensional ability parameters are presented the procedure is a generalization of a procedure by albert 1992 for estimating the two parameter normal ogive model the procedure supports analyzing data from multiple populations and incomplete designs it is shown that restrictions can be imposed on the factor matrix for testing specific hypotheses about the ability structure the technique is illustrated using simulated and real data

music like doa estimation without estimating the number of sources this paper presents a new direction of arrival doa estimation algorithm where doa estimation is realized in the framework of beamforming by introducing a new optimization problem the new optimization problem aims to minimize the variance of the array output subject to retaining the output power at the look direction and precluding a trivial solution the proposed algorithm is equivalent to the multiple signal classification music algorithm in some special cases it is therefore named as the music like method however different from the classical music algorithm the new method is not based on subspace decomposition and the number of sources is hence not required for direction finding this enables us to avoid performance deterioration caused by incorrect source number estimation the effectiveness and advantage of the proposed algorithm is theoretically investigated and computational complexity is also addressed to verify the theoretical analysis computer simulations are implemented and comparisons with other algorithms are made

marginal maximum likelihood estimation of item parameters application of an em algorithm maximum likelihood estimation of item parameters in the marginal distribution integrating over the distribution of ability becomes practical when computing procedures based on an em algorithm are used by characterizing the ability distribution empirically arbitrary assumptions about its form are avoided the em procedure is shown to apply to general item response models lacking simple sufficient statistics for ability this includes models with more than one latent dimension

a distributed framework for dimensionality reduction and denoising with the recent upsurge in the amount of data transmission and use in networks a lot of emphasis is placed on designing efficient methods to extract useful information by reducing the size of large data sets a major approach in data dimensionality reduction relies on the estimation of the principal data covariance eigenspace in this paper a distributed algorithmic framework is put forth for finding the principal eigenspace of spatially scattered sensor data toward this end the standard principal component analysis framework is reformulated as a separable constrained minimization problem which is solved by utilizing coordinate descent techniques combined with the alternating direction method of multipliers computationally simple local updating recursions are obtained that involve only single hop intersensor communications the proposed distributed algorithm is shown to be robust even in the presence of inter sensor communication noise and converge to the principal eigenspace when inter sensor links are ideal the proposed distributed framework is used in a data denoising setting while extensive numerical tests using both synthetic and real data demonstrate a fast convergence rate better steady state performance of the novel algorithm over existing alternatives and communication noise resilience

a cross associative neural network for svd of non squared data matrix in signal processing this paper proposes a cross associative neural network cann for singular value decomposition svd of a non squared data matrix in signal processing in order to improve the convergence speed and avoid the potential instability of the deterministic networks associated with the cross correlation neural network models we study the global asymptotic stability of the network for tracking all the singular components and show that the selection of its learning rate in the iterative algorithm is independent of the singular value distribution of a non squared matrix the performances of cann are shown via simulations

least squares type algorithm for adaptive implementation of pisarenko s harmonic retrieval method pisarenko s harmonic retrieval method involves determining the minimum eigenvalue and the corresponding eigenvector of the covariance matrix of the observed random process recently thompson 9 suggested a constrained gradient search procedure for obtaining an adaptive version of pisarenko s method and his simulations have verified that the frequency estimates provided by his procedure were unbiased however the main cost of this technique was that the initial convergence rate could be very slow for certain poor initial conditions restating the constrained minimization as an unconstrained nonlinear problem we derived an alternative gauss newton type recursive algorithm which also used the second derivative matrix or hessian this algorithm may also be viewed as an approximate least squares algorithm simulations have been performed to compare this algorithm to a slight variant of thompson s original algorithm the most important conclusions are that the least squares type algorithm has faster convergence in the beginning while its convergence rate close to the true parameters depends on the signal to noise ratio of the input signal the approximate least squares algorithm resolves the sinusoids much faster than the gradient version

convergence of learning algorithms with constant learning rates the behavior of neural network learning algorithms with a small constant learning rate epsilon in stationary random input environments is investigated it is rigorously established that the sequence of weight estimates can be approximated by a certain ordinary differential equation in the sense of weak convergence of random processes as epsilon tends to zero as applications backpropagation in feedforward architectures and some feature extraction algorithms are studied in more detail

stability behavior for unsupervised learning abstract we consider models proposed by oja karhunen cox and adams for a task of unsupervised learning questions of existence of solutions and stability behavior of those solutions are addressed we also discuss the impact of temporary connections in the stability of the learning process

algorithms for nonnegative independent component analysis we consider the task of solving the independent component analysis ica problem x as given observations x with a constraint of nonnegativity of the source random vector s we refer to this as nonnegative independent component analysis and we consider methods for solving this task for independent sources with nonzero probability density function pdf p s down to s 0 it is sufficient to find the orthonormal rotation y wz of prewhitened sources z vx which minimizes the mean squared error of the reconstruction of z from the rectified version y sup of y we suggest some algorithms which perform this both based on a nonlinear principal component analysis pca approach and on a geodesic search method driven by differential geometry considerations we demonstrate the operation of these algorithms on an image separation problem which shows in particular the fast convergence of the rotation and geodesic methods and apply the approach to a musical audio analysis task

asymptotic performance analysis of subspace adaptive algorithms introduced in the neural network literature in the neural network literature many algorithms have been proposed for estimating the eigenstructure of covariance matrices we first show that many of these algorithms when presented in a common framework show great similitudes with the gradient like stochastic algorithms usually encountered in the signal processing literature we derive the asymptotic distribution of these different recursive subspace estimators a closed form expression of the covariances in distribution of eigenvectors and associated projection matrix estimators are given and analyzed in particular closed form expressions of the mean square error of these estimators are given it is found that these covariance matrices have a structure very similar to those describing batch estimation techniques the accuracy of our asymptotic analysis is checked by numerical simulations and it is found to be valid not only for a small step size but in a very large domain finally convergence speed and deviation from orthonormality of the different algorithms are compared and several tradeoffs are analyzed

global convergence of oja s subspace algorithm for principal component extraction oja s principal subspace algorithm is a well known and powerful technique for learning and tracking principal information in time series a thorough investigation of the convergence property of oja s algorithm is undertaken in this paper the asymptotic convergence rates of the algorithm is discovered the dependence of the algorithm on its initial weight matrix and the singularity of the data covariance matrix is comprehensively addressed

a general family of limited information goodness of fit statistics for multinomial data maydeu olivares and joe j am stat assoc 100 1009 1020 2005 psychometrika 71 713 732 2006 introduced classes of chi square tests for sparse multidimensional multinomial data based on low order marginal proportions our extension provides general conditions under which quadratic forms in linear functions of cell residuals are asymptotically chi square the new statistics need not be based on margins and can be used for one dimensional multinomials we also provide theory that explains why limited information statistics have good power regardless of sparseness we show how quadratic form statistics can be constructed that are more powerful than x2 and yet have approximate chi square null distribution in finite samples with large models examples with models for truncated count data and binary item response data are used to illustrate the theory

performance analysis of adaptive eigenanalysis algorithms we present a rigorous analysis of several popular forms of short memory adaptive eigenanalysis algorithms using a stochastic averaging method a first order analysis shows that the algorithms do not have any equilibrium points despite published claims to the contrary through averaging analysis we show that they hover around an appropriate eigenvector a second order analysis is also given without the gaussian noise assumption and our results greatly outperform an earlier approximation in the literature the second order analysis has been of much interest in the offline study but in the dynamic adaptive case is uncommon

a note on linear function approximation using random projections linear function approximations based on random projections are proposed and justified for a class of fixed point and minimization problems

a composite likelihood inference in latent variable models for ordinal longitudinal responses the paper proposes a composite likelihood estimation approach that uses bivariate instead of multivariate marginal probabilities for ordinal longitudinal responses using a latent variable model the model considers time dependent latent variables and item specific random effects to be accountable for the interdependencies of the multivariate ordinal items time dependent latent variables are linked with an autoregressive model simulation results have shown composite likelihood estimators to have a small amount of bias and mean square error and as such they are feasible alternatives to full maximum likelihood model selection criteria developed for composite likelihood estimation are used in the applications furthermore lower order residuals are used as measures of fit for the selected models

modification indices for the 2 pl and the nominal response model in this paper it is shown that various violations of the 2 pl model and the nominal response model can be evaluated using the lagrange multiplier test or the equivalent efficient score test the tests presented here focus on violation of local stochastic independence and insufficient capture of the form of the item characteristic curves primarily the tests are item oriented diagnostic tools but taken together they also serve the purpose of evaluation of global model fit a useful feature of lagrange multiplier statistics is that they are evaluated using maximum likelihood estimates of the null model only that is the parameters of alternative models need not be estimated as numerical examples an application to real data and some power studies are presented

how should we assess the fit of rasch type models approximating the power of goodness of fit statistics in categorical data analysis we investigate the performance of three statistics r 1 r 2 glas in psychometrika 53 525 546 1988 and m 2 maydeu olivares joe in j am stat assoc 100 1009 1020 2005 psychometrika 71 713 732 2006 to assess the overall fit of a one parameter logistic model 1pl estimated by marginal maximum likelihood ml r 1 and r 2 were specifically designed to target specific assumptions of rasch models whereas m 2 is a general purpose test statistic we report asymptotic power rates under some interesting violations of model assumptions different item discrimination presence of guessing and multidimensionality as well as empirical rejection rates for correctly specified models and some misspecified models all three statistics were found to be more powerful than pearson s x 2 against two and three parameter logistic alternatives 2pl and 3pl and against multidimensional 1pl models the results suggest that there is no clear advantage in using goodness of fit statistics specifically designed for rasch type models to test these models when marginal ml estimation is used

estimation of the eigenvalues of ab for a 0 b 0 abstract for positive definite hermitian matrices a and b we obtain an upper and a lower bound for the eigenvalues of ab in terms of those of a and b

limited information goodness of fit testing in multidimensional contingency tables we introduce a family of goodness of fit statistics for testing composite null hypotheses in multidimensional contingency tables these statistics are quadratic forms in marginal residuals up to order r they are asymptotically chi square under the null hypothesis when parameters are estimated using any asymptotically normal consistent estimator for a widely used item response model when r is small and multidimensional tables are sparse the proposed statistics have accurate empirical type i errors unlike pearson s x2 for this model in nonsparse situations the proposed statistics are also more powerful than x2 in addition the proposed statistics are asymptotically chi square when applied to subtables and can be used for a piecewise goodness of fit assessment to determine the source of misfit in poorly fitting models

the dynamics of on line principal component analysis the learning dynamics of an on line algorithm for principal component analysis is described exactly in the thermodynamic limit by means of coupled ordinary differential equations for a set of order parameters it is demonstrated that learning is delayed significantly because existing symmetries among student vectors have to be broken a closely related effect is the perfect or partial loss of initial knowledge in the course of learning the analysis shows that different learning rates for the student vectors improve the performance of the algorithm drastically

generalized residuals for general models for contingency tables with application to item response theory generalized residuals are a tool employed in the analysis of contingency tables to examine possible sources of model error they have typically been applied to log linear models and to latent class models a general approach to generalized residuals is developed for a very general class of models for contingency tables to illustrate their use generalized residuals are applied to models based on item response theory irt models such models are commonly applied to analysis of standardized achievement or aptitude tests to obtain a realistic perspective on application of generalized residuals actual testing data are employed

distributed estimation and control of algebraic connectivity over random graphs in this paper we propose a distributed algorithm for the estimation and control of the connectivity of ad hoc networks in the presence of a random topology first given a generic random graph we introduce a novel stochastic power iteration method that allows each node to estimate and track the algebraic connectivity of the underlying expected graph using results from stochastic approximation theory we prove that the proposed method converges almost surely a s to the desired value of connectivity even in the presence of imperfect communication scenarios the estimation strategy is then used as a basic tool to adapt the power transmitted by each node of a wireless network in order to maximize the network connectivity in the presence of realistic medium access control mac protocols or simply to drive the connectivity toward a desired target value numerical results corroborate our theoretical findings thus illustrating the main features of the algorithm and its robustness to fluctuations of the network graph due to the presence of random link failures

on stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix abstract in applications of signal processing and pattern recognition eigenvectors and eigenvalues of the statistical mean of a random matrix sequence are needed iterative methods are suggested and analyzed in which no sample moments are used convergence is shown by stochastic approximation theory

on the sampling theory roundations of item response theory models item response theory it models are now in common use for the analysis of dichotomous item responses this paper examines the sampling theory foundations for statistical inference in these models the discussion includes some history on the stochastic subject versus the random sampling interpretations of the probability in irt models the relationship between three versions of maximum likelihood estimation for irt models estimating versus estimating predictors irt models and loglinear models the identifiability of irt models and the role of robustness and bayesian statistics from the sampling theory perspective

on the relationship between item response theory and factor analysis of discretized variables equivalence of marginal likelihood of the two parameter normal ogive model in item response theory irt and factor analysis of dichotomized variables fa was formally proved the basic result on the dichotomous variables was extended to multicategory cases both ordered and unordered categorical data pair comparison data arising from multiple judgment sampling were discussed as a special case of the unordered categorical data a taxonomy of data for the irt and fa models was also attempted

pairwise residuals and diagnostic tests for misspecified dependence structures in models for binary longitudinal data maximum likelihood estimation of models for binary longitudinal data is typically inconsistent if the dependence structure is misspecified unfortunately diagnostics specifically designed for detecting such misspecifications are scant we develop residuals and diagnostic tests based on comparing observed and expected frequencies of response patterns over time in the presence of arbitrary time varying and time invariant covariates to overcome the sparseness problem we use lower order marginal tables such as two way tables for pairs of time points aggregated over covariate patterns our proposed pairwise concordance residuals are valuable for exploratory diagnostics and for constructing both generic tests for misspecified dependence structure as well as targeted adjacent pair concordance tests for excess serial dependence the proposed methods are straightforward to implement and work well for general situations regardless of the number of time points and the number and types of covariates

chaotic diffusion on periodic orbits the perturbed arnold cat map chaotic diffusion on periodic orbits pos is studied for the perturbed arnold cat map on a cylinder in a range of perturbation parameters corresponding to an extended structural stability regime of the system on the torus the diffusion coefficient is calculated using the following po formulas 1 the curvature expansion of the ruelle zeta function 2 the average of the po winding number squared w 2 weighted by a stability factor 3 the uniform nonweighted average of w 2 the results from formulas 1 and 2 agree very well with those obtained by standard methods for all the perturbation parameters considered formula 3 gives reasonably accurate results for sufficiently small parameters corresponding also to cases of a considerably nonuniform hyperbolicity this is due to uniformity sum rules satisfied by the po lyapunov eigenvalues at fixed w these sum rules follow from general arguments and are supported by much numerical evidence

goodness of fit for generalized linear latent variables models generalized linear latent variables models gllvm enable the modeling of relationships between manifest and latent variables where the manifest variables are distributed according to a distribution of the exponential family e g binomial or normal and to the multinomial distribution for ordinal manifest variables these models are widely used in social sciences to test the appropriateness of a particular model one needs to define a goodness of fit test statistic gfi in the normal case one can use a likelihood ratio test or a modified version proposed by satorra and bentler 2001 s b gfi that compares the sample covariance matrix to the estimated covariance matrix induced by the model in the binary case pearson type test statistics can be used if the number of observations is sufficiently large in the other cases including the case of mixed types of manifest variables there exists gfi based on a comparison between a pseudo sample covariance and the model covariance of the manifest vari

robust recursive least squares learning algorithm for principal component analysis a learning algorithm for the principal component analysis pca is developed based on the least square minimization the dual learning rate parameters are adjusted adaptively to make the proposed algorithm capable of fast convergence and high accuracy for extracting all principal components the proposed algorithm is robust to the error accumulation existing in the sequential pca algorithm we show that all information needed for pca can he completely represented by the unnormalized weight vector which is updated based only on the corresponding neuron input output product the updating of the normalized weight vector can be referred to as a leaky hebb s rule the convergence of the proposed algorithm is briefly analyzed we also establish the relation between oja s rule and the least squares learning rule finally the simulation results are given to illustrate the effectiveness of this algorithm for pca and tracking time varying directions of arrival

periodic orbits and chaotic diffusion probability distributions abstract periodic orbit po formulas for chaotic diffusion probability distributions pds are examined in the case of the perturbed arnol d cat map on the cylinder this translationally invariant system exhibits a transition from uniform to nonuniform hyperbolicity as the perturbation parameter is increased two coarse grained pds describing the diffusion between unit cells of the system are studied a a pd based on po ensembles b a pd based on generic ensembles the approximate po formula for pd b gives results which fluctuate around the expected gaussian distribution for all parameters considered and thus agree qualitatively with results from standard methods the exact po formula for pd a gives similar results only for sufficiently small parameters the results for large parameters decrease monotonically relative to the gaussian distribution this deviation seems to disappear as the po period is increased