karhunen loeve expansion of a set of rotated templates we propose a novel method for efficiently calculating the eigenvectors of uniformly rotated images of a set of templates as we show the images can be optimally approximated by a linear series of eigenvectors which can be calculated without actually decomposing the sample covariance matrix

a blind approach to hammerstein model identification this paper discusses the hammerstein model identification using a blind approach by fast sampling at the output it is shown that identification of the linear part can be achieved based only on the output measurements that makes the hammerstein model identification possible without knowing the structure of the nonlinearity and the internal variables

an instrumental least squares support vector machine for nonlinear system identification least squares support vector machines ls svms originating from statistical learning and reproducing kernel hilbert space rkhs theories represent a promising approach to identify nonlinear systems via nonparametric estimation of the involved nonlinearities in a computationally and stochastically attractive way however application of ls svms and other rkhs variants in the identification context is formulated as a regularized linear regression aiming at the minimization of the 2 loss of the prediction error this formulation corresponds to the assumption of an auto regressive noise structure which is often found to be too restrictive in practical applications in this paper instrumental variable iv based estimation is integrated into the ls svm approach providing under minor conditions consistent identification of nonlinear systems regarding the noise modeling error it is shown how the cost function of the ls svm is modified to achieve an iv based solution although a practically well applicable choice of the instrumental variable is proposed for the derived approach optimal choice of this instrument in terms of the estimates associated variance still remains to be an open problem the effectiveness of the proposed iv based ls svm scheme is also demonstrated by a monte carlo study based simulation example

adaptive digital control of hammerstein nonlinear systems with limited output sampling this paper is motivated by the practical control considerations that nonlinearity is abundant in industrial processes and output sampling rates are often limited due to hardware constraints in particular for a hammerstein nonlinear sampled data system in which the output sampling period is an integer multiple of the input updating period we derive by using a polynomial transformation technique a mathematical model which is suitable for parameter estimation with dual rate measurement data further we present an adaptive control scheme for such a dual rate nonlinear system the parameter estimation based adaptive algorithm can achieve virtually asymptotically optimal control and ensure that the closed loop system is stable and globally convergent the simulation results are included

neural field dynamics under variation of local and global connectivity and finite transmission speed abstract spatially continuous networks with heterogeneous connections are ubiquitous in biological systems in particular neural systems to understand the mutual effects of locally homogeneous and globally heterogeneous connectivity we investigate the stability of the steady state activity of a neural field as a function of its connectivity the variation of the connectivity is implemented through manipulation of a heterogeneous two point connection embedded into the otherwise homogeneous connectivity matrix and by variation of the connectivity strength and transmission speed detailed examples including the ginzburg landau equation and various other local architectures are discussed our analysis shows that developmental changes such as the myelination of the cortical large scale fiber system generally result in the stabilization of steady state activity independent of the local connectivity non oscillatory instabilities are shown to be independent of any influences of time delay

maximum likelihood estimation via the ecm algorithm a general framework two major reasons for the popularity of the em algorithm are that its maximum step involves only complete data maximum likelihood estimation which is often computationally simple and that its convergence is stable with each iteration increasing the likelihood when the associated complete data maximum likelihood estimation itself is complicated em is less attractive because the m step is computationally unattractive in many cases however complete data maximum likelihood estimation is relatively simple when conditional on some function of the parameters being estimated

adaptive multiregression in reproducing kernel hilbert spaces the multiaccess mimo channel case this paper introduces a wide framework for online i e time adaptive supervised multiregression tasks the problem is formulated in a general infinite dimensional reproducing kernel hilbert space rkhs in this context a fairly large number of nonlinear multiregression models fall as special cases including the linear case any convex continuous and not necessarily differentiable function can be used as a loss function in order to quantify the disagreement between the output of the system and the desired response the only requirement is the subgradient of the adopted loss function to be available in an analytic form to this end we demonstrate a way to calculate the subgradients of robust loss functions suitable for the multiregression task as it is by now well documented when dealing with online schemes in rkhs the memory keeps increasing with each iteration step to attack this problem a simple sparsification strategy is utilized which leads to an algorithmic scheme of linear complexity with respect to the number of unknown parameters a convergence analysis of the technique based on arguments of convex analysis is also provided to demonstrate the capacity of the proposed method the multiregressor is applied to the multiaccess multiple input multiple output channel equalization task for a setting with poor resources and nonavailable channel information numerical results verify the potential of the method when its performance is compared with those of the state of the art linear techniques which in contrast use space time coding more antenna elements as well as full channel information

regression models for identifying noise sources in magnetic resonance images stochastic noise susceptibility artifacts magnetic field and radiofrequency inhomogeneities and other noise components in magnetic resonance images mris can introduce serious bias into any measurements made with those images we formally introduce three regression models including a rician regression model and two associated normal models to characterize stochastic noise in various magnetic resonance imaging modalities including diffusion weighted imaging dwi and functional mri fmri estimation algorithms are introduced to maximize the likelihood function of the three regression models we also develop a diagnostic procedure for systematically exploring mr images to identify noise components other than simple stochastic noise and to detect discrepancies between the fitted regression models and mri data the diagnostic procedure includes goodness of fit statistics measures of influence and tools for graphical display the goodness of fit statistics can assess the key assumptions of the three r

identification of state dependent parameter models with support vector regression a support vector regression approach is presented for the identification of state dependent parameter arx models whose parameters are described as functions of past inputs and outputs the problem of identifying the state dependent parameters reduces to a standard support vector regression problem with a kernel function which is defined in terms of the kernels used to represent the individual parameters numerical examples show that the support vector method gives accurate parameter estimates for systems which have a state dependent parameter representation

an optimal two stage identification algorithm for hammerstein wiener nonlinear systems abstract an optimal two stage identification algorithm is presented for hammerstein wiener systems where two static nonlinear elements surround a linear block the proposed algorithm consists of two steps the first one is the recursive least squares and the second one is the singular value decomposition of two matrices whose dimensions are fixed and do not increase as the number of the data point increases moreover the algorithm is shown to be convergent in the absence of noise and convergent with probability one in the presence of white noise

recursive identification of multi input multi output errors in variables hammerstein systems the note considers the identification of multi input multi output errors in variables hammerstein systems in which both the input and output can be observed but with additive noises being arma processes with unknown coefficients with the help of stochastic approximation combined with the deconvolution kernel function the recursive algorithms are proposed for estimating coefficients of the linear subsystem and for the values of the nonlinear function under some reasonable conditions all the estimates are proved to converge to the true values with probability one these results include identification of the errors in variables linear systems as a special case a simulation example is given justifying the theoretical analysis

an imputation regularized optimization algorithm for high dimensional missing data problems and beyond missing data are frequently encountered in high dimensional problems but they are usually difficult to deal with by using standard algorithms such as the expectation maximization algorithm and its variants to tackle this difficulty some problem specific algorithms have been developed in the literature but there still lacks a general algorithm this work is to fill the gap we propose a general algorithm for high dimensional missing data problems the algorithm works by iterating between an imputation step and a regularized optimization step at the imputation step the missing data are imputed conditionally on the observed data and the current estimates of parameters and at the regularized optimization step a consistent estimate is found via the regularization approach for the minimizer of a kullback leibler divergence defined on the pseudocomplete data for high dimensional problems the consistent estimate can be found under sparsity constraints the consistency of the averaged estimate for the true parameter can be established under quite general conditions the algorithm is illustrated by using high dimensional gaussian graphical models high dimensional variable selection and a random coefficient model

on robustness of marginal regression coefficient estimates and hazard functions in multivariate survival analysis of family data when the frailty distribution is mis specified the shared frailty model is an extension of the cox model to correlated failure times and essentially a random effects model for failure time outcomes in this model the latent frailty shared by individual members in a cluster acts multiplicatively as a factor on the hazard function and is typically modelled parametrically one commonly used distribution is gamma where both shape and scale parameters are set to be the same to allow for unique identification of baseline hazard function it is popular because it is a conjugate prior and the posterior distribution possesses the same form as gamma in addition the parameter can be interpreted as a time independent cross ratio function a natural extension of odds ratio to failure time outcomes in this paper we study the effect of frailty distribution mis specification on the marginal regression estimates and hazard functions under assumed gamma distribution with an application to family studies the simulation results show that the biases are generally 10 and lower even when the true frailty distribution deviates substantially from the assumed gamma distribution this suggests that the gamma frailty model can be a practical choice in real data analyses if the regression parameters and marginal hazard function are of primary interest and individual cluster members are exchangeable with respect to their dependencies

semiparametric maximum likelihood for measurement error model regression this paper presents an em algorithm for semiparametric likelihood analysis of linear generalized linear and nonlinear regression models with measurement errors in explanatory variables a structural model is used in which probability distributions are specified for a the response and b the measurement error a distribution is also assumed for the true explanatory variable but is left unspecified and is estimated by nonparametric maximum likelihood for various types of extra information about the measurement error distribution the proposed algorithm makes use of available routines that would be appropriate for likelihood analysis of a and b if the true x were available simulations suggest that the semiparametric maximum likelihood estimator retains a high degree of efficiency relative to the structural maximum likelihood estimator based on correct distributional assumptions and can outperform maximum likelihood based on an incorrect distributional assumption the approach is illustrated on three examples with a variety of structures and types of extra information about the measurement error distribution

bifurcation analysis of neural mass models impact of extrinsic inputs and dendritic time constants neural mass models nmms explain dynamics of neuronal populations and were designed to strike a balance between mathematical simplicity and biological plausibility they are currently widely used as generative models for noninvasive electrophysiological brain measurements that is magneto and electroencephalography m eeg here we systematically describe the oscillatory regimes which a nmm of a single cortical source with extrinsic input from other cortical and subcortical areas to each subpopulation can explain for this purpose we used bifurcation analysis to describe qualitative changes in system behavior in response to quantitative input changes this approach allowed us to describe sequences of oscillatory regimes given some specific input trajectory we systematically classified these sequential phenomena and mapped them into parameter space our analysis suggests a principled scheme of how complex m eeg phenomena can be modeled parsimoniously on two time scales while the system displays fast oscillations it slowly traverses phase space to another qualitatively different oscillatory regime depending on the input dynamics the resulting scheme is useful for applications where one needs to model an ordered sequence of switching between qualitatively different oscillatory regimes for example in pharmacological interventions epilepsy sleep or context induced state changes

bayesian robust multivariate linear regression with incomplete data abstract the multivariate t distribution and other normal independent multivariate distributions such as the multivariate slash distribution and the multivariate contaminated distribution are used for robust regression with complete or incomplete data most previous work focused on the method of maximum likelihood estimation for linear regression using normal independent distributions this article considers bayesian estimation of multivariate linear regression models using normal independent distributions with fully observed predictor variables and possible missing values from outcome variables a monotone data augmentation algorithm for posterior simulation of the parameters and missing data imputation is presented the posterior distributions of functions of the parameters can be obtained using monte carlo methods the monotone data augmentation algorithm can also be used for creating multiple imputations for incomplete data sets an illustrative example of using the multivariate t is also included

identification of block oriented nonlinear systems starting from linear approximations a survey abstract block oriented nonlinear models are popular in nonlinear system identification because of their advantages of being simple to understand and easy to use many different identification approaches were developed over the years to estimate the parameters of a wide range of block oriented nonlinear models one class of these approaches uses linear approximations to initialize the identification algorithm the best linear approximation framework and the approximation framework or equivalent frameworks allow the user to extract important information about the system guide the user in selecting good candidate model structures and orders and prove to be a good starting point for nonlinear system identification algorithms this paper gives an overview of the different block oriented nonlinear models that can be identified using linear approximations and of the identification algorithms that have been developed in the past a non exhaustive overview of the most important other block oriented nonlinear system identification approaches is also provided throughout this paper

general formulas for obtaining the mles and the asymptotic variance covariance matrix in mapping quantitative trait loci when using the em algorithm we present in this paper general formulas for deriving the maximum likelihood estimates and the asymptotic variance covariance matrix of the positions and effects of quantitative trait loci qtls in a finite normal mixture model when the em algorithm is used for mapping qtls the general formulas are based on two matrices d and q where d is the genetic design matrix characterizing the genetic effects of the qtls and q is the conditional probability matrix of qtl genotypes given flanking marker genotypes containing the information on qtl positions with the general formulas it is relatively easy to extend qtl mapping analysis to using multiple marker intervals simultaneously for mapping multiple qtls for analyzing qtl epistasis and for estimating the heritability of quantitative traits simulations were performed to evaluate the performance of the estimates of the asymptotic variances of qtl positions and effects

estimation and model selection for an ide based spatio temporal model a state space model of the stochastic spatio temporal integro difference equation ide is derived based on multidimensional sampling theory the dimensions of the state space and parameter space of the model are identified from the spatial bandwidth of the system and the support of the redistribution kernel of the ide when both the bandwidth and the kernel support are unknown a method to propose a number of state space and parameter space dimensions is presented these chosen dimensions result in a number of candidate model structures bayesian model selection making use of bayes factor the data augmentation algorithm and importance sampling is then used to identify the model best suited to represent the data in a maximum a posteriori sense

a theory for multiresolution signal decomposition the wavelet representation multiresolution representations are effective for analyzing the information content of images the properties of the operator which approximates a signal at a given resolution were studied it is shown that the difference of information between the approximation of a signal at the resolutions 2 sup j 1 and 2 sup j where j is an integer can be extracted by decomposing this signal on a wavelet orthonormal basis of l sup 2 r sup n the vector space of measurable square integrable n dimensional functions in l sup 2 r a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi x this decomposition defines an orthogonal multiresolution representation called a wavelet representation it is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters wavelet representation lies between the spatial and fourier domains for images the wavelet representation differentiates several spatial orientations the application of this representation to data compression in image coding texture discrimination and fractal analysis is discussed

estimation of adjusted rate differences using additive negative binomial regression rate differences are an important effect measure in biostatistics and provide an alternative perspective to rate ratios when the data are event counts observed during an exposure period adjusted rate differences may be estimated using an identity link poisson generalised linear model also known as additive poisson regression a problem with this approach is that the assumption of equality of mean and variance rarely holds in real data which often show overdispersion an additive negative binomial model is the natural alternative to account for this however standard model fitting methods are often unable to cope with the constrained parameter space arising from the non negativity restrictions of the additive model in this paper we propose a novel solution to this problem using a variant of the expectation conditional maximisation either algorithm our method provides a reliable way to fit an additive negative binomial regression model and also permits flexible generalisations using semi parametric regression functions we illustrate the method using a placebo controlled clinical trial of fenofibrate treatment in patients with type ii diabetes where the outcome is the number of laser therapy courses administered to treat diabetic retinopathy an r package is available that implements the proposed method copyright 2016 john wiley sons ltd

stability and bifurcations in neural fields with finite propagation speed and general connectivity a stability analysis is presented for neural field equations in the presence of finite propagation speed along axons and for a general class of connectivity kernels and synaptic proper ties sufficient conditions are given for the stability of equilibrium solutions it is shown that the propagation delays play a significant role in nonstationary bifurcations of equilibria whereas the stationary bifurcations depend only on the connectivity kernel in the case of nonstationary bifurca tions bounds are determined on the frequencies of the resulting oscillatory solutions a perturbative scheme is used to calculate the types of bifurcations leading to spatial patterns oscillations and traveling waves for high propagation speeds a simple method is derived that allows the determi nation of the bifurcation type by visual inspection of the fourier transforms of the kernel and its first moment results are numerically illustrated on a class of neurologically plausible systems with combinations of gaussian excitatory and inhibitory connections

nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci estimation of the covariance structure of longitudinal processes is a fundamental prerequisite for the practical deployment of functional mapping designed to study the genetic regulation and network of quantitative variation in dynamic complex traits we present a nonparametric approach for estimating the covariance structure of a quantitative trait measured repeatedly at a series of time points specifically we adopt huang et al s 2006 biometrika 93 85 98 approach of invoking the modified cholesky decomposition and converting the problem into modeling a sequence of regressions of responses a regularized covariance estimator is obtained using a normal penalized likelihood with an l 2 penalty this approach embedded within a mixture likelihood framework leads to enhanced accuracy precision and flexibility of functional mapping while preserving its biological relevance simulation studies are performed to reveal the statistical properties and advantages of the proposed method a real example from a mouse genome project is analyzed to illustrate the utilization of the methodology the new method will provide a useful tool for genome wide scanning for the existence and distribution of quantitative trait loci underlying a dynamic trait important to agriculture biology and health sciences

an extended mallows model for ranked data aggregation in this article we study the rank aggregation problem which aims to find a consensus ranking by aggregating multiple ranking lists to address the problem probabilistically we formulate an elabo

penalized maximum likelihood image reconstruction using space alternating generalized em algorithms most expectation maximization em type algorithms for penalized maximum likelihood image reconstruction converge slowly particularly when one incorporates additive background effects such as scatter random coincidences dark current or cosmic radiation in addition regularizing smoothness penalties or priors introduce parameter coupling rendering intractable the m steps of most em type algorithms this paper presents space alternating generalized em sage algorithms for image reconstruction which update the parameters sequentially using a sequence of small hidden data spaces rather than simultaneously using one large complete data space the sequential update decouples the m step so the maximization can typically be performed analytically we introduce new hidden data spaces that are less informative than the conventional complete data space for poisson data and that yield significant improvements in convergence rate this acceleration is due to statistical considerations not numerical overrelaxation methods so monotonic increases in the objective function are guaranteed we provide a general global convergence proof for sage methods with nonnegativity constraints

isotropic polyharmonic b splines scaling functions and wavelets in this paper we use polyharmonic b splines to build multidimensional wavelet bases these functions are nonseparable multidimensional basis functions that are localized versions of radial basis functions we show that rabut s elementary polyharmonic b splines do not converge to a gaussian as the order parameter increases as opposed to their separable b spline counterparts therefore we introduce a more isotropic localization operator that guarantees this convergence resulting into the isotropic polyharmonic b splines next we focus on the two dimensional quincunx subsampling scheme this configuration is of particular interest for image processing because it yields a finer scale progression than the standard dyadic approach however up until now the design of appropriate filters for the quincunx scheme has mainly been done using the mcclellan transform in our approach we start from the scaling functions which are the polyharmonic b splines and as such explicitly known and we derive a family of polyharmonic spline wavelets corresponding to different flavors of the semi orthogonal wavelet transform e g orthonormal b spline and dual the filters are automatically specified by the scaling relations satisfied by these functions we prove that the isotropic polyharmonic b spline wavelet converges to a combination of four gabor atoms which are well separated in the frequency domain we also show that these wavelets are nearly isotropic and that they behave as an iterated laplacian operator at low frequencies we describe an efficient fast fourier transform based implementation of the discrete wavelet transform based on polyharmonic b splines

identification of discrete hammerstein systems using kernel regression estimates in this note a discrete time hammerstein system is identified the weighting function of the dynamical subsystem is recovered by the correlation method the main results concern estimation of the nonlinear memoryless subsystem no conditions concerning functional form of the transform characteristic of the subsystem are made and an algorithm for estimation of the characteristic is presented the algorithm is a nonparametric kernel estimate of regression functions calculated from dependent data it is shown that the algorithm converges to the characteristic as the number of observations tend to infinity for sufficiently smooth characteristics the rate of convergence is o n 2 5 in probability

data driven spatio temporal modeling using the integro difference equation a continuous in space discrete in time dynamic spatio temporal model known as the integro difference equation ide model is presented in the context of data driven modeling a novel decomposition of the ide is derived leading to state space representation that does not couple the number of states with the number of observation locations or the number of parameters based on this state space model an expectation maximization em algorithm is developed in order to jointly estimate the ide model s spatial field and spatial mixing kernel the resulting modeling framework is demonstrated on a set of examples

the design and use of steerable filters the authors present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters allowing one to adaptively steer a filter to any orientation and to determine analytically the filter output as a function of orientation steerable filters may be designed in quadrature pairs to allow adaptive control over phase as well as orientation the authors show how to design and steer the filters and present examples of their use in the analysis of orientation and phase angularly adaptive filtering edge detection and shape from shading one can also build a self similar steerable pyramid representation the same concepts can be generalized to the design of 3 d steerable filters

latent variable models for mixed discrete and continuous outcomes we propose a latent variable model for mixed discrete and continuous outcomes the model accommodates any mixture of outcomes from an exponential family and allows for arbitrary covariate effects as well as direct modelling of covariates on the latent variable an em algorithm is proposed for parameter estimation and estimates of the latent variables are produced as a by product of the analysis a generalized likelihood ratio test can be used to test the significance of covariates affecting the latent outcomes this method is applied to birth defects data where the outcomes of interest are continuous measures of size and binary indicators of minor physical anomalies infants who were exposed in utero to anticonvulsant medications are compared with controls

computing steerable principal components of a large set of images and their rotations we present here an efficient algorithm to compute the principal component analysis pca of a large image set consisting of images and for each image the set of its uniform rotations in the plane we do this by pointing out the block circulant structure of the covariance matrix and utilizing that structure to compute its eigenvectors we also demonstrate the advantages of this algorithm over similar ones with numerical experiments although it is useful in many settings we illustrate the specific application of the algorithm to the problem of cryo electron microscopy

finite model order accuracy in hammerstein model estimation hammerstein models is one of the most commonly used model classes used for identifying nonlinear systems a static input nonlinearity followed by a linear dynamical part is an adequate way to model many real life systems this paper investigates the asymptotic in terms of sample size variance of hammerstein model estimates the work extends earlier results by ninness and gibson 2002 in the following ways not only frequency function estimation but estimation of general quantities is considered the expressions are not restricted to be valid asymptotically in the model order in addition the results cover model structures having noise models and allow for data generated under feedback the increase in variance due to the estimation of the input nonlinearity is characterized in particular under open loop operation white additive noise and the assumption of a separable process it is shown that the variance increase is exactly a term that was observed in ninness and gibson 2002 to result in good agreement with simulations this term vanishes in the formal asymptotic in model order analysis in ninness and gibson 2002

a method for assessing age time disease incidence using serial prevalence data summary this paper considers nonparametric estimation of age and time specific trends in disease incidence using serial prevalence data collected from multiple cross sectional samples of a population over time the methodology accounts for differential selection of diseased and undiseased individuals resulting for example from differences in mortality it is shown that when a log linear incidence odds model is adopted an em algorithm provides a convenient method for carrying out maximum likelihood estimation primarily using existing generalized linear models software the procedure is quite general allowing a range of age time incidence models to be fitted under the same framework furthermore by making use of existing software for fitting generalized additive models the procedure can be generalized with virtually no extra complexity to allow maximization of a penalized likelihood for smooth nonparametric estimation automatic choice of smoothing level for the penalized likelihood estimates is discussed using generalized cross validation the method is applied to a data set on serial toxoplasmosis prevalence which has previously been analyzed under the assumption of nondifferential selection a variety of age time incidence models are fitted and the sensitivity to plausible differential selection patterns is considered it is found that nonmultiplicative models are unnecessary and that qualitative incidence trends are fairly robust to differential selection

subspace identification of hammerstein systems using least squares support vector machines this paper presents a method for the identification of multiple input multiple output mimo hammerstein systems for the goal of prediction the method extends the numerical algorithms for subspace state space system identification n4sid mainly by rewriting the oblique projection in the n4sid algorithm as a set of componentwise least squares support vector machines ls svms regression problems the linear model and static nonlinearities follow from a low rank approximation of a matrix obtained from this regression problem

maximum likelihood estimation of nonlinear structural equation models the existing maximum likelihood theory and its computer software in structural equation modeling are established based on linear relationships among manifest variables and latent variables however models with nonlinear relationships are often encountered in social and behavioral sciences in this article an em type algorithm is developed for maximum likelihood estimation of a general nonlinear structural equation model to avoid computation of the complicated multiple integrals involved the e step is completed by a metropolis hastings algorithm it is shown that the m step can be completed efficiently by simple conditional maximization standard errors of the maximum likelihood estimates are obtained via louis s formula the methodology is illustrated with results from a simulation study and two real examples

a derivation of a macroscopic field theory of the brain from the quasi microscopic neural dynamics abstract the purpose of this paper is twofold first we present a semi quantitative nonlinear field theory of the brain under realistic anatomical connectivity conditions describing the interaction between functional units within the brain this macroscopic field theory is derived from the quasi microscopic conversion properties of neural populations occurring at synapses and somas the quasi microscopic models by wilson cowan 1972 1973 and nunez 1974 can be derived from these functional units are treated as inhomogeneities within a nonlinear one dimensional neural tissue second for the case of the kelso experiment the field equation is treated analytically and numerically and can be reduced to a set of ordinary differential equations which corresponds to a model by jirsa et al 1994 1995 this phenomenological model reproduces the spatio temporal phenomena experimentally observed here the most prominent property of the neural tissue is the parametric excitation the macroscopic field parameters can be expressed by quasi microscopic neural parameters

regression analysis with linked data record linkage or exact matching can be used to join together two files that contain information on the same individuals but lack unique personal identification codes the possibility of errors in linkage causes problems for estimating the relationships between variables on the two files the effect is analogous to the impact of measurement error a model of a linear regression relationship between variables in linked files is proposed assuming the probabilities that pairs of records are links are known an unbiased estimator of the regression coefficients is derived methods for estimating the linkage probabilities by using mixture models are discussed a consistent estimator of the covariance matrix of the proposed estimator is proposed a bootstrap estimator is used to reflect the impact of the uncertainty in record linkage model parameters on the estimators of the regression parameters a simulation study compares the performance of the proposed estimator and alternatives

hammerstein system identification through best linear approximation inversion and regularisation abstracthammerstein systems are composed by the cascading of a static nonlinearity and a linear system in this paper a methodology for identifying such systems using a combination of least squares support vector machines ls svm and best linear approximation bla techniques is proposed to do this a novel method for estimating the intermediate variable is presented allowing a clear separation of the identification steps first an approximation to the linear block is obtained through the bla of the system then an approximation to the intermediate variable is obtained using the inversion of the estimated linear block and the known output afterwards a nonlinear model is calculated through ls svm using the estimated intermediate variable and the known input to do this the regularisation capabilities of ls svm play a crucial role finally a parametric re estimation of the linear block is made the method was tested in three examples two of them with hard nonlinearities and was compared with four

space alternating generalized expectation maximization algorithm the expectation maximization em method can facilitate maximizing likelihood functions that arise in statistical estimation problems in the classical em paradigm one iteratively maximizes the conditional log likelihood of a single unobservable complete data space rather than maximizing the intractable likelihood function for the measured or incomplete data em algorithms update all parameters simultaneously which has two drawbacks 1 slow convergence and 2 difficult maximization steps due to coupling when smoothness penalties are used the paper describes the space alternating generalized em sage method which updates the parameters sequentially by alternating between several small hidden data spaces defined by the algorithm designer the authors prove that the sequence of estimates monotonically increases the penalized likelihood objective derive asymptotic convergence rates and provide sufficient conditions for monotone convergence in norm two signal processing applications illustrate the method estimation of superimposed signals in gaussian noise and image reconstruction from poisson measurements in both applications the sage algorithms easily accommodate smoothness penalties and converge faster than the em algorithms

optimal approximation of uniformly rotated images relationship between karhunen loeve expansion and discrete cosine transform we present the concept that for uniformly rotated images the optimal approximation of the images can be obtained by computing the basis vectors for the discrete cosine transform dct of the original image in polar coordinates and that the images can be represented as linear combinations of the basis vectors

improving on deterministic approximate bayesian inferences for mixture distributions this paper presents the branching approach which can improve deterministic implementation methods such as variational bayesian inference and expectation propagation method for bayesian mixture distributions this proposed approach utilizes a set of artificial conditions defined by using the latent variables of the mixture distribution this condition set is updated iteratively by branching based on a condition selected from the previous condition set the target approximate bayesian inference is obtained by merging the approximate conditional inferences under each condition in the condition set the proposed approach is compared with several standard implementation methods by using a numerical example and a real world example

neural network approach for identification of hammerstein systems in this paper four different on line gradient based learning algorithms for training neural network hammerstein models are presented in a unified framework these algorithms namely the backpropagation for series parallel models the backpropagation sensitivity method and truncated backpropagation through time algorithm bptt for parallel models are derived analysed and compared for the truncated bptt it is shown that determination of the number of unfolding time steps necessary to calculate the gradient with an assumed degree of accuracy can be made on the basis of impulse response functions of sensitivity models the algorithms are shown to differ in their computational complexity gradient approximation accuracy and convergence rates numerical examples are also included to compare the performance of the algorithms

sparse kalman filtering approaches to realized covariance estimation from high frequency financial data estimation of the covariance matrix of asset returns from high frequency data is complicated by asynchronous returns market microstructure noise and jumps one technique for addressing both asynchronous returns and market microstructure is the kalman expectation maximization kem algorithm however the kem approach assumes log normal prices and does not address jumps in the return process which can corrupt estimation of the covariance matrix in this paper we extend the kem algorithm to price models that include jumps we propose a sparse kalman filtering approach to this problem in particular we develop a kalman expectation conditional maximization algorithm to determine the unknown covariance as well as detecting the jumps in order to promote a sparse estimate of the jumps we consider both laplace and the spike and slab jump priors numerical results using simulated data show that each of these approaches provide for improved covariance estimation relative to the kem method in a variety of settings where jumps occur

identification scheme for hammerstein output error models with bounded noise this technical note presents a method for the identification of hammerstein output error hoe models corrupted by bounded noise the method is based on an iterative scheme using two optimal bounding ellipsoid obe type algorithms the proposed approach allows the joint estimation of the linear part and of the nonlinear part with a low computational burden a numerical example is given so as to show the efficiency of the method

subspace identification methods for hammerstein systems rank constraint and dimension problem this article studies the subspace identification methods sims for hammerstein systems with major focus on a rank constraint and the related dimension problem we analyse the effects of the rank constraint on the three steps of a unifying framework for sims the rank constraint has no effect on the first two steps but does so on the third step if the rank constraint is ignored as in the existing over parametrised method opm for hammerstein system identification the optimality of the resulting estimate can still be established even so the opm may suffer from the dimension problem resulting in a low numerical efficiency to resolve the dimension problem we propose a new subspace based method named as the least parametrised method lpm for identification of hammerstein systems with non coupling input nonlinearities simulation results are provided to demonstrate the effectiveness of the lpm and show the necessity of considering the rank constraint to improve the numerical efficiency

large sample joint posterior approximations when full conditionals are approximately normal modern bayesian statistical methods such as gibbs and metropolis hastings sampling were developed to liberate statisticians from the necessity of making large sample assumptions and to facilitate the numerical approximation of problems that had previously been analytically intractable counter to this trend we develop a method for constructing asymptotic joint posterior approximations based on models with k blocks of parameters and where the corresponding properly normalized full conditionals are themselves asymptotically normal we illustrate these techniques by applying them to particular linear and generalized linear mixed models glmms we also consider the relevance of different parameterizations with regard to our asymptotics recent work has indicated that gibbs samplers based on so called centering parameterizations result in better convergence properties for the resulting markov chains our results for the one way random effects model shed some light on this issue for this example we also

iterative automated record linkage using mixture models the goal of record linkage is to link quickly and accurately records that correspond to the same person or entity whereas certain patterns of agreements and disagreements on variables are more likely among records pertaining to a single person than among records for different people the observed patterns for pairs of records can be viewed as arising from a mixture of matches and nonmatches mixture model estimates can be used to partition record pairs into two or more groups that can be labeled as probable matches links and probable nonmatches nonlinks a method is proposed and illustrated that uses marginal information in the database to select mixture models identifies sets of records for clerks to review based on the models and marginal information incorporates clerically reviewed data as they become available into estimates of model parameters and classifies pairs as links nonlinks or in need of further clerical review the procedure is illustrated with five datasets from the u s bureau

identification of mimo hammerstein models using least squares support vector machines this paper studies a method for the identification of hammerstein models based on least squares support vector machines ls svms the technique allows for the determination of the memoryless static nonlinearity as well as the estimation of the model parameters of the dynamic arx part this is done by applying the equivalent of bai s overparameterization method for identification of hammerstein systems in an ls svm context the siso as well as the mimo identification cases are elaborated the technique can lead to significant improvements with respect to classical overparameterization methods as illustrated in a number of examples another important advantage is that no stringent assumptions on the nature of the nonlinearity need to be imposed except for a certain degree of smoothness

identifying mimo hammerstein systems in the context of subspace model identification methods in this paper we outline the extension of the moesp family of subspace model identification schemes to the hammerstein type of nonlinear system two types of identification problem are considered the first type assumes the polynomial structure of the static nonlinearity to be given and the task is to identify both the linear system dynamics and the unknown proportional constants in the para metrization of the static nonlinearity the second type addresses the identification of both the linear dynamic part and the static nonlinearity where only limited a priori information regarding the structure of the nonlinearity is available the improved robustness properties of the algorithms developed for this second type of hammerstein identification problem over existing correlation based schemes is illustrated by a numerical example

nonlinear black box models in system identification mathematical foundations we discuss several aspects of the mathematical foundations of the nonlinear black box identification problem we shall see that the quality of the identification procedure is always a result of a certain trade off between the expressive power of the model we try to identify the larger the number of parameters used to describe the model the more flexible is the approximation and the stochastic error which is proportional to the number of parameters a consequence of this trade off is the simple fact that a good approximation technique can be the basis of a good identification algorithm from this point of view we consider different approximation methods and pay special attention to spatially adaptive approximants we introduce wavelet and neuron approximations and show that they are spatially adaptive then we apply the acquired approximation experience to estimation problems finally we consider some implications of these theoretical developments for the practically implemented versions of the spatially adaptive algorithms

on compactly supported spline wavelets and a duality principle let v 1 v 0 v 1 be a multiresolution analysis of l 2 generated by the mth order b spline n m x we exhibit a compactly supported basic wavelet m x that generates the corresponding orthogonal complementary wavelet subspaces w 1 w 0 w 1 consequently the two finite sequences that describe the two scale relations of n m x and m x in terms of n m 2x j j z yield an efficient reconstruction algorithm to give an efficient wavelet decomposition algorithm based on these two finite sequences we derive a duality principle which also happens to yield the dual bases n m x j and 59 m x j relative to n m x j and m x j respectively

nonparametric identification of wiener systems by orthogonal series a wiener system i e a system comprising a linear dynamic and a nonlinear memoryless subsystems connected in a cascade is identified both the input signal and disturbance are random white and gaussian the unknown nonlinear characteristic is strictly monotonous and differentiable and therefore the problem of its recovering from input output observations of the whole system is nonparametric it is shown that the inverse of the characteristic is a regression function and a class of orthogonal series nonparametric estimates recovering the regression is proposed and analyzed the estimates apply the trigonometric legendre and hermite orthogonal functions pointwise consistency of all the algorithms is shown under some additional smoothness restrictions the rates of their convergence are examined and compared an algorithm to identify the impulse response of the linear subsystem is proposed

extending the state space model to accommodate missing values in responses and covariates this article proposes an extended state space model for accommodating multivariate panel data the novel aspect of this contribution is an adjustment to the classical model for multiple subjects that allows missingness in the covariates in addition to the responses missing covariate data are handled by a second state space model nested inside the first to represent unobserved exogenous information relevant kalman filter equations are derived and explicit expressions are provided for both the e and m steps of an expectation maximization em algorithm to obtain maximum gaussian likelihood estimates of all model parameters in the presence of missing data the resulting em algorithm becomes computationally intractable but a simplification of the m step leads to a new procedure that is shown to be an expectation conditional maximization ecm algorithm under exogeneity of the covariates simulation studies reveal that the approach appears to be relatively robust to moderate percentages of missing dat

mixtures of multivariate power exponential distributions summary an expanded family of mixtures of multivariate power exponential distributions is introduced while fitting heavy tails and skewness have received much attention in the model based clustering literature recently we investigate the use of a distribution that can deal with both varying tail weight and peakedness of data a family of parsimonious models is proposed using an eigen decomposition of the scale matrix a generalized expectation maximization algorithm is presented that combines convex optimization via a minorization maximization approach and optimization based on accelerated line search algorithms on the stiefel manifold lastly the utility of this family of models is illustrated using both toy and benchmark data

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

dynamic reconstruction of chaotic systems from inter spike intervals using least squares support vector machines this work presents a methodology for dynamic reconstruction of chaotic systems from inter spike interval isi time series obtained via integrate and fire if models in this methodology least squares support vector machines lssvms have been employed for approximating the dynamic behaviors of the systems under investigation higher generalization capability and avoidance of local minima constitute the main reasons behind the choice of lssvms as the approximation tool simulation results have shown that established lssvm models possess great potential for the reconstruction of chaotic dynamics in other words they are able to estimate some dynamic invariants of the underlying chaotic systems as well as they can accurately predict short term evolution within the horizon of predictability moreover lssvm models maintain their reconstruction performance even in the case of the existence of noisy data

recursive identification of hammerstein systems convergence rate and asymptotic normality in this work recursive identification algorithms are developed for hammerstein systems under the conditions considerably weaker than those in the existing literature for example orders of linear subsystems may be unknown and no specific conditions are imposed on their moving average part the recursive algorithms for estimating both linear and nonlinear parts are based on stochastic approximation and kernel functions almost sure convergence and strong convergence rates are derived for all estimates in addition the asymptotic normality of the estimates for the nonlinear part is also established the nonlinearity considered in the paper is more general than those discussed in the previous papers a numerical example verifies the theoretical analysis with simulation results

bilinear probabilistic principal component analysis probabilistic principal component analysis ppca is a popular linear latent variable model for performing dimension reduction on 1 d data in a probabilistic manner however when used on 2 d data such as images ppca suffers from the curse of dimensionality due to the subsequently large number of model parameters to overcome this problem we propose in this paper a novel probabilistic model on 2 d data called bilinear ppca bppca this allows the establishment of a closer tie between bppca and its nonprobabilistic counterpart moreover two efficient parameter estimation algorithms for fitting bppca are also developed experiments on a number of 2 d synthetic and real world data sets show that bppca is more accurate than existing probabilistic and nonprobabilistic dimension reduction methods

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

identification of a class of nonlinear autoregressive models with exogenous inputs based on kernel machines in this paper we propose a new approach to identify a new class of nonlinear autoregressive models with exogenous inputs narx based on kernel machine and space projection kmsp the well known hammerstein wiener model which includes blocks of nonlinear static functions in series with a linear dynamic block is a subset of the narx models considered in the kmsp based approach kernel machine is used to represent the functions and space projection to separate the represented functions we also discuss two possible ambiguities and give conditions to avoid such ambiguities the asymptotic behavior of the proposed approach is analyzed the performance of the proposed method is verified by simulation studies

globally convergent algorithms for maximizing a likelihood function summary in this paper we show global convergence under very general assumptions of iterative maximization procedures with cyclic fixing of groups of parameters maximizing over the remaining parameters further we show that a slightly modified newton procedure applied to the derivative of the reciprocal likelihood function in a one dimensional exponential family is globally convergent we also prove that if the distribution of the sufficient statistic is infinitely divisible then the newton method applied to the derivative of the log likelihood function converges by combining these results we obtain general globally convergent iterative procedures for maximizing the likelihood function in any regular k dimensional exponential family the likelihood function can be maximized either by successively maximizing over one parameter at a time or by maximizing in the newton direction

leptokurtic and platykurtic class of robust symmetrical and asymmetrical time series models abstract in this study we examined the well known autoregressive time series model in which innovations follow the flexible class of two piece distributions based on the scale mixtures of normal tp smn family the mentioned class of distributions is a rich class of distributions family that covers the robust symmetric asymmetric light heavy tailed distributions a key feature of this study is using a new suitable hierarchical representation of the family to obtain the maximum likelihood estimates of the model parameters via an expectation maximization em type algorithm performances and usefulness of the proposed model and estimates are shown via simulation studies and a real dataset

spatiotemporal multi resolution approximation of the amari type neural field model neural fields are spatially continuous state variables described by integro differential equations which are well suited to describe the spatiotemporal evolution of cortical activations on multiple scales here we develop a multi resolution approximation mra framework for the integro difference equation ide neural field model based on semi orthogonal cardinal b spline wavelets in this way a flexible framework is created whereby both macroscopic and microscopic behavior of the system can be represented simultaneously state and parameter estimation is performed using the expectation maximization em algorithm a synthetic example is provided to demonstrate the framework

nonlinear black box modeling in system identification a unified overview a nonlinear black box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics there has been considerable recent interest in this area with structures based on neural networks radial basis networks wavelet networks and hinging hyperplanes as well as wavelet transform based methods and models based on fuzzy sets and fuzzy rules this paper describes all these approaches in a common framework from a user s perspective it focuses on what are the common features in the different approaches the choices that have to be made and what considerations are relevant for a successful system identification application of these techniques it is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space these mappings are discussed separately the latter mapping is usually formed as a basis function expansion the basis functions are typically formed from one simple scalar function which is modified in terms of scale and location the expansion from the scalar argument to the regressor space is achieved by a radial or a ridge type approach basic techniques for estimating the parameters in the structures are criterion minimization as well as two step procedures where first the relevant basis functions are determined using data and then a linear least squares step to determine the coordinates of the function approximation a particular problem is to deal with the large number of potentially necessary parameters this is handled by making the number of used parameters considerably less than the number of offered parameters by regularization shrinking pruning or regressor selection

effective algorithms with circulant block matrices abstract effective numerical algorithms for circulant block matrices a whose blocks are circulant are obtained the eigenvalues of such matrices are determined in terms of the eigenvalues of matrices of reduced dimension and systems of linear equations involving these matrices are solved efficiently using fast fourier transforms

brief paper convergence of the iterative algorithm for a general hammerstein system identification the convergence of the iterative identification algorithm for a general hammerstein system has been an open problem for a long time in this paper it is shown that the convergence can be achieved by incorporating a regularization procedure on the nonlinearity in addition to a normalization step on the parameters

estimating and using propensity scores with partially missing data abstract investigators in observational studies have no control over treatment assignment as a result large differences can exist between the treatment and control groups on observed covariates which can lead to badly biased estimates of treatment effects propensity score methods are an increasingly popular method for balancing the distribution of the covariates in the two groups to reduce this bias for example using matching or subclassification sometimes in combination with model based adjustment to estimate propensity scores which are the conditional probabilities of being treated given a vector of observed covariates we must model the distribution of the treatment indicator given these observed covariates much work has been done in the case where covariates are fully observed we address the problem of calculating propensity scores when covariates can have missing values in such cases which commonly arise in practice the pattern of missing covariates can be prognostically important and

spatiotemporal system identification with continuous spatial maps and sparse estimation we present a framework for the identification of spatiotemporal linear dynamical systems we use a state space model representation that has the following attributes 1 the number of spatial observation locations are decoupled from the model order 2 the model allows for spatial heterogeneity 3 the model representation is continuous over space and 4 the model parameters can be identified in a simple and sparse estimation procedure the model identification procedure we propose has four steps 1 decomposition of the continuous spatial field using a finite set of basis functions where spatial frequency analysis is used to determine basis function width and spacing such that the main spatial frequency contents of the underlying field can be captured 2 initialization of states in closed form 3 initialization of state transition and input matrix model parameters using sparse regression the least absolute shrinkage and selection operator method and 4 joint state and parameter estimation using an iterative kalman filter sparse regression algorithm to investigate the performance of the proposed algorithm we use data generated by the kuramoto model of spatiotemporal cortical dynamics the identification algorithm performs successfully predicting the spatiotemporal field with high accuracy whilst the sparse regression leads to a compact model

robust maximum likelihood estimation of multivariable dynamic systems this paper examines the problem of estimating linear time invariant state space system models in particular it addresses the parametrization and numerical robustness concerns that arise in the multivariable case these difficulties are well recognised in the literature resulting for example in extensive study of subspace based techniques as well as recent interest in data driven local co ordinate approaches to gradient search solutions the paper here proposes a different strategy that employs the expectation maximisation em technique the consequence is an algorithm that is iterative with associated likelihood values that are locally convergent to stationary points of the gaussian likelihood function furthermore theoretical and empirical evidence presented here establishes additional attractive properties such as numerical robustness avoidance of difficult parametrization choices the ability to naturally and easily estimate non zero initial conditions and moderate computational cost moreover since the methods here are maximum likelihood based they have associated known and asymptotically optimal statistical properties

unbalanced repeated measures models with structured covariance matrices the question of how to analyze unbalanced or incomplete repeated measures data is a common problem facing analysts we address this problem through maximum likelihood analysis using a general linear model for expected responses and arbitrary structural models for the within subject covariances models that can be fit include standard univariate and multivariate models with incomplete data random effects models and models with time series and factor analytic error structures we describe newton raphson and fisher scoring algorithms for computing maximum likelihood estimates and generalized em algorithms for computing restricted and unrestricted maximum likelihood estimates an example fitting several models to a set of growth data is included

diagonalizing properties of the discrete cosine transforms since its introduction in 1974 by ahmed et al the discrete cosine transform dct has become a significant tool in many areas of digital signal processing especially in signal compression there exist eight types of discrete cosine transforms dcts we obtain the eight types of dcts as the complete orthonormal set of eigenvectors generated by a general form of matrices in the same way as the discrete fourier transform dft can be obtained as the eigenvectors of an arbitrary circulant matrix these matrices can be decomposed as the sum of a symmetric toeplitz matrix plus a hankel or close to hankel matrix scaled by some constant factors we also show that all the previously proposed generating matrices for the dcts are simply particular cases of these general matrix forms using these matrices we obtain for each dct a class of stationary processes verifying certain conditions with respect to which the corresponding dct has a good asymptotic behavior in the sense that it approaches karhunen loeve transform performance as the block size n tends to infinity as a particular result we prove that the eight types of dcts are asymptotically optimal for all finite order markov processes we finally study the decorrelating power of the dcts obtaining expressions that show the decorrelating behavior of each dct with respect to any stationary processes

semiparametric probit models with univariate and bivariate current status data summary multivariate current status data are frequently encountered in biomedical and public health studies semiparametric regression models have been extensively studied for univariate current status data but most existing estimation procedures are computationally intensive involving either penalization or smoothing techniques it becomes more challenging for the analysis of multivariate current status data in this article we study the maximum likelihood estimations for univariate and bivariate current status data under the semiparametric probit regression models we present a simple computational procedure combining the expectation maximization algorithm with the pool adjacent violators algorithm for solving the monotone constraint on the baseline function asymptotic properties of the maximum likelihood estimators are investigated including the calculation of the explicit information bound for univariate current status data as well as the asymptotic consistency and convergence rate for bivariate current status data extensive simulation studies showed that the proposed computational procedures performed well under small or moderate sample sizes we demonstrate the estimation procedure with two real data examples in the areas of diabetic and hiv research

using em to obtain asymptotic variance covariance matrices the sem algorithm abstract the expectation maximization em algorithm is a popular and often remarkably simple method for maximum likelihood estimation in incomplete data problems one criticism of em in practice is that asymptotic variance covariance matrices for parameters e g standard errors are not automatic byproducts as they are when using some other methods such as newton raphson in this article we define and illustrate a procedure that obtains numerically stable asymptotic variance covariance matrices using only the code for computing the complete data variance covariance matrix the code for em itself and code for standard matrix operations the basic idea is to use the fact that the rate of convergence of em is governed by the fractions of missing information to find the increased variability due to missing information to add to the complete data variance covariance matrix we call this supplemented em algorithm the sem algorithm theory and particular examples reinforce the conclusion that the sem alg

on the series expansion approach to the identification of hammerstein systems a polynomial identification algorithm for recovering a nonlinearity in the hammerstein system is proposed the estimate employs the legendre orthogonal system with adaptively selected number of terms the global consistency along with rates of convergence are established no assumptions concerning continuity of the nonlinearity or its functional form are made a data driven method using the cross validation technique for selecting the number of terms in the estimate is presented

limitations of feedback properties imposed by open loop right half plane poles an integral constraint on the closed loop transfer function of unstable open loop systems is used to quantify a tradeoff between feedback properties of the closed loop system this results in a lower bound on the peak of the complementary sensitivity function for unstable plants the lower bound illustrates the nature of the tradeoff that is imposed and provides insight into the difficulties imposed on the control of unstable systems it can be used to quantitatively evaluate tradeoffs between the frequency of the open loop unstable pole the peak of the complementary sensitivity functions and the bandwidth of the closed loop system such a relationship has long been recognized and is embodied in the classical design rule of thumb which states that the closed loop bandwidth must be at least twice as large as the frequency of the unstable pole results which give insight into this rule and a theoretical basis for it are provided

brief paper revisiting hammerstein system identification through the two stage algorithm for bilinear parameter estimation the two stage algorithm tsa has been extensively used and adapted for the identification of hammerstein systems it is essentially based on a particular formulation of hammerstein systems in the form of bilinearly parameterized linear regressions this paper has been motivated by a somewhat contradictory fact though the optimality of the tsa has been established by bai in 1998 only in the case of some special weighting matrices the unweighted tsa is usually used in practice it is shown in this paper that the unweighted tsa indeed gives the optimal solution of the weighted nonlinear least squares problem formulated with a particular weighting matrix this provides a theoretical justification of the unweighted tsa and also leads to a generalization of the obtained result to the case of colored noise with noise whitening numerical examples of identification of hammerstein systems are presented to validate the theoretical analysis

biometrika centenary theory and general methodology note reprinted in biometrika one hundred years edited by d m titterington and d r cox oxford university press 11 50 reference stat article 2001 002doi 10 1093 biomet 88 1 13view record in web of science record created on 2006 04 21 modified on 2017 05 12

the discrete cosine transform each discrete cosine transform dct uses n real basis vectors whose components are cosines in the dct 4 for example the j th component of boldv k is cos j frac 1 2 k frac 1 2 frac pi n these basis vectors are orthogonal and the transform is extremely useful in image processing if the vector boldx gives the intensities along a row of pixels its cosine series sum c k boldv k has the coefficients c k boldx boldv k n they are quickly computed from a fast fourier transform but a direct proof of orthogonality by calculating inner products does not reveal how natural these cosine vectors are we prove orthogonality in a different way each dct basis contains the eigenvectors of a symmetric second difference matrix by varying the boundary conditions we get the established transforms dct 1 through dct 4 other combinations lead to four additional cosine transforms the type of boundary condition dirichlet or neumann centered at a meshpoint or a midpoint determines the applications that are appropriate for each transform the centering also determines the period n 1 or n in the established transforms n frac 1 2 or n frac 1 2 in the other four the key point is that all these eigenvectors of cosines come from simple and familiar matrices