minimum distance estimation in random coefficient regression models random coefficient regression models are important in modeling heteroscedastic multivariate linear regression in econometrics the analysis of panel data is one example in statistics the random and mixed effects models of anova deconvolution models and affine mixture models are all special cases of random coefficient regression some inferential problems such as constructing prediction regions for the modeled response require a good nonparametric estimator of the unknown coefficient distribution this paper introduces and studies a consistent nonparametric minimum distance method for estimating the coefficient distribution our estimator translates the difficult problem of estimating an inverse radon transform into a minimization problem

local linear regression smoothers and their minimax efficiencies in this paper we introduce a smooth version of local linear regression estimators and address their advantages the mse and mise of the estimators are computed explicitly it turns out that the local linear regression smoothers have nice sampling properties and high minimax efficiency they are not only efficient in rates but also nearly efficient in constant factors in the nonparametric regression context the asymptotic minimax lower bound is developed via the heuristic of the hardest onedimensional subproblem of donoho and liu connections of the minimax risk with the modulus of continuity are made the lower bound is also applicable for estimating conditional mean regression and conditional quantiles for both fixed and random design regression problems

optimal rates of convergence for deconvolving a density abstract suppose that the sum of two independent random variables x and z is observed where z denotes measurement error and has a known distribution and where the unknown density f of x is to be estimated one application is the estimation of a prior density for a sequence of location parameters a second application arises in the errors in variables problem for nonlinear and generalized linear models when one attempts to model the distribution of the true but unobservable covariates this article shows that if z is normally distributed and f has k bounded derivatives then the fastest attainable convergence rate of any nonparametric estimator of f is only log n k 2 therefore deconvolution with normal errors may not be a practical proposition other error distributions are also treated stefanski carroll 1987a estimators achieve the optimal rates the results given have versions for multiplicative errors where they imply that even optimal rates are exceptionally slow

reducing variance in univariate smoothing a variance reduction technique in nonparametric smoothing is proposed at each point of estimation form a linear combination of a preliminary estimator evaluated at nearby points with the coefficients specified so that the asymptotic bias remains unchanged the nearby points are chosen to maximize the variance reduction we study in detail the case of univariate local linear regression while the new estimator retains many advantages of the local linear estimator it has appealing asymptotic relative efficiencies bandwidth selection rules are available by a simple constant factor adjustment of those for local linear estimation a simulation study indicates that the finite sample relative efficiency often matches the asymptotic relative efficiency for moderate sample sizes this technique is very general and has a wide range of applications

efficient estimation and inferences for varying coefficient models abstract this article deals with statistical inferences based on the varying coefficient models proposed by hastie and tibshirani local polynomial regression techniques are used to estimate coefficient functions and the asymptotic normality of the resulting estimators is established the standard error formulas for estimated coefficients are derived and are empirically tested a goodness of fit test technique based on a nonparametric maximum likelihood ratio type of test is also proposed to detect whether certain coefficient functions in a varying coefficient model are constant or whether any covariates are statistically significant in the model the null distribution of the test is estimated by a conditional bootstrap method our estimation techniques involve solving hundreds of local likelihood equations to reduce the computational burden a one step newton raphson estimator is proposed and implemented the resulting one step procedure is shown to save computational cost on an order of tens with no

orthogonal eigensubspace estimation using neural networks we present a neural network nn approach for simultaneously estimating all or some of the orthogonal eigenvectors of a symmetric nonindefinite matrix corresponding to its repeated minimum in magnitude eigenvalue this problem has its origin in the constrained minimization framework and has extensive applications in signal processing we recast this problem into the nn framework by constructing an appropriate energy function which the nn minimizes the nn is of feedback type with the neurons having sigmoidal activation function the proposed approach is analyzed to characterize the nature of the minimizers of the energy function the main result is that the matrix w is a minimizer of the energy function if and only if the columns of w are the orthogonal eigenvectors with a given norm corresponding to the smallest eigenvalue of the given matrix further all minimizers are global minimizers bounds on the integration time step that is required to numerically solve the system of differential equations which define the dynamics of the nn have also been derived results of computer simulations are presented to support our analysis

efficient statistical inference procedures for partially nonlinear models and their applications motivated by an analysis of a real data set in ecology we consider a class of partially nonlinear models where both a nonparametric component and a parametric component are present we develop two new estimation procedures to estimate the parameters in the parametric component consistency and asymptotic normality of the resulting estimators are established we further propose an estimation procedure and a generalized f test procedure for the nonparametric component in the partially nonlinear models asymptotic properties of the newly proposed estimation procedure and the test statistic are derived finite sample performance of the proposed inference procedures are assessed by monte carlo simulation studies an application in ecology is used to illustrate the proposed methods

a one step smoothing newton method for second order cone programming a new smoothing function for the second order cone programming is given by smoothing the symmetric perturbed fischer burmeister function based on this new function a one step smoothing newton method is presented for solving the second order cone programming the proposed algorithm solves only one linear system of equations and performs only one line search at each iteration this algorithm does not have restrictions regarding its starting point and is q quadratically convergent numerical results suggest the effectiveness of our algorithm

a trust region filter method for general non linear programming filter approach is initially proposed by fletcher and leyffer in 2002 because of promising numerical results filter methods are recently attached importance to if the objective function value or the constraint violation is reduced this step is accepted by a filter which is the basic idea of the filter in this paper the filter technique is employed in a trust region algorithm in every trial step the step length is controlled by a trust region radius moreover our purpose is not to reduce the objective function and constraint violation directly to overcome some bad cases we aim to reduce the degree of constraint violation and the entry of some function and the function is a combination of the objective function and the degree of constraint violation the algorithm in this paper requires neither lagrangian multipliers nor strong decrease condition in certain conditions this method produces k t points for the original problem moreover maratos effect can be avoided for our algorithm

an improved sequential quadratic programming algorithm for solving general nonlinear programming problems abstract in this paper a class of general nonlinear programming problems with inequality and equality constraints is discussed firstly the original problem is transformed into an associated simpler equivalent problem with only inequality constraints then inspired by the ideals of the sequential quadratic programming sqp method and the method of system of linear equations sle a new type of sqp algorithm for solving the original problem is proposed at each iteration the search direction is generated by the combination of two directions which are obtained by solving an always feasible quadratic programming qp subproblem and a sle respectively moreover in order to overcome the maratos effect the higher order correction direction is obtained by solving another sle the two sles have the same coefficient matrices and we only need to solve the one of them after a finite number of iterations by a new line search technique the proposed algorithm possesses global and superlinear convergence under some suitable assumptions without the strict complementarity finally some comparative numerical results are reported to show that the proposed algorithm is effective and promising

multiple emitter location and signal parameter estimation processing the signals received on an array of sensors for the location of the emitter is of great enough interest to have been treated under many special case assumptions the general problem considers sensors with arbitrary locations and arbitrary directional characteristics gain phase polarization in a noise interference environment of arbitrary covariance matrix this report is concerned first with the multiple emitter aspect of this problem and second with the generality of solution a description is given of the multiple signal classification music algorithm which provides asymptotically unbiased estimates of 1 number of incident wavefronts present 2 directions of arrival doa or emitter locations 3 strengths and cross correlations among the incident waveforms 4 noise interference strength examples and comparisons with methods based on maximum likelihood ml and maximum entropy me as well as conventional beamforming are included an example of its use as a multiple frequency estimator operating on time series is included

curve estimation when the design density is low in problems where a high dimensional design is projected into a lower number of dimensions the density of the new design is typically not bounded away from zero over its support even if the original one was contexts where this problem arises include projection pursuit regression estimation in single index models and application of the projection slice method of radon transform inversion theoretical work in these settings typically involves ignoring data toward the ends of the support of the projected design but in practice that waste of information is not an attractive option motivated by these difficulties we analyze the way in which local linear smoothing is affected by unboundedly sparse design and apply the conclusions of that study to develop empirical adaptive bandwidth choice methods our results even add to knowledge in the familiar case of a design density that is bounded away from zero where they provide adaptive bandwidth selectors that are optimal right to the ends of the design interval

finite sample variance of local polynomials analysis and solutions abstract fitting local polynomials in nonparametric regression has a number of advantages the attractive theoretical features are in a partial contradiction to variance properties for random design and to practical experience over a broad range of situations no upper bound can be given for the conditional variance the unconditional variance is infinite when using optimal weights with compact support properties are better for gaussian weights we analyze local polynomials for finite sample size both theoretically and numerically it turns out that difficulties arise in sparse regions in the realization of the design when the realization has locally a small variance and or a skew empirical distribution two small sample modifications of local polynomials are presented local increase of bandwidth in sparse regions of the design and local polynomial ridge regression both modifications combine a good finite sample behavior with the asymptotic advantages of local polynomials

using local correlation in kernel based smoothers for dependent data nous considerons le probleme general de lisser des donnees correlees afin d obtenir une estimation non parametrique d une fonction quand un nombre aleatoire mais borne de mesures est disponible pour chaque sujet nous proposons une simple extension du lissage par regression polynomiale qui conserve les proprietes asymptotiques du modele de travail d independance tout en reduisant le biais conditionnel et la variance pour des tailles d echantillons courantes comme le montrent des calculs exacts dans certains modeles particuliers nous illustrons notre methode par le lissage de donnees longitudinales de declin fonctionnel chez 100 patients atteints de la maladie de huntington on montre que la classe des polynomes locaux d estimation par noyaux precedemment consideres dans la litterature utilise la structure de correlation de maniere apparemment non optimale ce qui explique pourquoi quelques tentatives anterieures d incorporer la correlation sont asymptotiquement inferieures a l hypothese de travail d independance

a dual purpose principal and minor subspace gradient flow the dual purpose principal and minor subspace gradient flow can be used to track principal subspace ps and if altered simply by the sign it can also serve as a minor subspace ms trackor this is of practical significance in the implementations of algorithms in this paper a unified information criterion is proposed and a dual purpose principal and minor subspace gradient flow is derived based on the information criterion in this dual purpose gradient flow the weight matrix length is self stabilizing i e moving towards unit length in each learning step the energy function associated with the dual purpose gradient flow for tracking ps and ms is given and it exhibits a unique global minimum attained if and only if its state matrices span the ps or ms of the autocorrelation matrix of a vector data stream the other stationary points of its energy function are unstable saddle points the proposed dual purpose gradient flow can efficiently track an orthonormal basis of the ps or ms which is illustrated through simulation experiments

estimating coefficient distributions in random coefficient regressions random coefficient regression models are important in representing linear models with heteroscedastic errors and in unifying the study of classical fixed effects and random effects linear models for prediction intervals and for bootstrapping in random coefficient regressions it is necessary to estimate the distributions of the random coefficients consistently we show that this is often possible and provide practical representative estimators of these distributions

extrinsic local regression on manifold valued data we propose an extrinsic regression framework for modeling data with manifold valued responses and euclidean predictors regression with manifold responses has wide applications in shape analysis neuroscience medical imaging and many other areas our approach embeds the manifold where the responses lie onto a higher dimensional euclidean space obtains a local regression estimate in that space and then projects this estimate back onto the image of the manifold outside the regression setting both intrinsic and extrinsic approaches have been proposed for modeling i i d manifold valued data however to our knowledge our work is the first to take an extrinsic approach to the regression problem the proposed extrinsic regression framework is general computationally efficient and theoretically appealing asymptotic distributions and convergence rates of the extrinsic regression estimates are derived and a large class of examples are considered indicating the wide applicability of our approach

a fast recursive total least squares algorithm for adaptive fir filtering this paper proposes a new fast recursive total least squares n rtls algorithm to recursively compute the tls solution for adaptive finite impulse response fir filtering the n rtls algorithm is based on the minimization of the constrained rayleigh quotient c rq in which the last entry of the parameter vector is constrained to the negative one as analysis results on the convergence of the proposed algorithm we study the properties of the stationary points of the c rq the high computational efficiency of the new algorithm depends on the efficient computation of the fast gain vector fgv and the adaptation of the c rq since the last entry of the parameter vector in the c rq has been fixed as the negative one a minimum point of the c rq is searched only along the input data vector and a more efficient n rtls algorithm is obtained by using the fgv as compared with davila s rtls algorithms the n rtls algorithm saves the 6m number of multiplies divides and square roots mads the global convergence of the new algorithm is studied by lasalle s invariance principle the performances of the relevant algorithms are compared via simulations and the long term numerical stability of the n rtls algorithm is verified

an efficient recursive total least squares algorithm for fir adaptive filtering an algorithm for recursively computing the total least squares tls solution to the adaptive filtering problem is described this algorithm requires o n multiplications per iteration to effectively track the n dimensional eigenvector associated with the minimum eigenvalue of an augmented sample covariance matrix it is shown that the recursive least squares rls algorithm generates biased adaptive filter coefficients when the filter input vector contains additive noise the tls solution on the other hand is seen to produce unbiased solutions examples of standard adaptive filtering applications that result in noise being added to the adaptive filter input vector are cited computer simulations comparing the relative performance of rls and recursive tls are described

on random tomography with unobservable projection angles we formulate and investigate a statistical inverse problem of a random tomographic nature where a probability density function on mathbb r 3 is to be recovered from observation of finitely many of its two dimensional projections in random and unobservable directions such a problem is distinct from the classic problem of tomography where both the projections and the unit vectors normal to the projection plane are observable the problem arises in single particle electron microscopy a powerful method that biophysicists employ to learn the structure of biological macromolecules strictly speaking the problem is unidentifiable and an appropriate reformulation is suggested hinging on ideas from kendall s theory of shape within this setup we demonstrate that a consistent solution to the problem may be derived without attempting to estimate the unknown angles if the density is assumed to admit a mixture representation

a robust sequential quadratic programming method the sequential quadratic programming method developed by wilson han and powell may fail if the quadratic programming subproblems become infeasible or if the associated sequence of search directions is unbounded this paper considers techniques which circumvent these difficulties by modifying the structure of the constraint region in the quadratic programming subproblems furthermore questions concerning the occurrence of an unbounded sequence of multipliers and problem feasibility are also addressed

monitoring profiles based on nonparametric regression methods the use statistical process control spc in monitoring and diagnosis of process and product quality profiles remains an important problem in various manufacturing industries the spc problem with a nonlinear profile is particularly challenging this article proposes a novel scheme to monitor changes in both the regression relationship and the variation of the profile online it integrates the multivariate exponentially weighted moving average procedure with the generalized likelihood ratio test based on nonparametric regression the proposed scheme not only provides an effective spc solution to handle nonlinear profiles which are common in industrial practice but it also resolves the latent problem in popular parametric monitoring methods of being unable to detect certain types of changes due to a misspecified out of control model our simulation results demonstrate the effectiveness and efficiency of the proposed monitoring scheme in addition a systematic diagnostic approach is provided to locate t

development and analysis of a neural network approach to pisarenko s harmonic retrieval method pisarenko s harmonic retrieval phr method is perhaps the first eigenstructure based spectral estimation technique the basic step in this method is the computation of eigenvector corresponding to the minimum eigenvalue of the autocorrelation matrix of the underlying data the authors recast a known constrained minimization formulation for obtaining this eigenvector into the neural network nn framework using the penalty function approach they develop an appropriate energy function for the nn this nn is of feedback type with the neurons having sigmoidal activation function analysis of the proposed approach shows that the required eigenvector is a minimizer with a given norm of this energy function further all its minimizers are global minimizers bounds on the integration time step that is required to numerically solve the system of nonlinear differential equations which define the network dynamics have been derived results of computer simulations are presented to support their analysis

a pattern search filter method for nonlinear programming without derivatives this paper formulates and analyzes a pattern search method for general constrained optimization based on filter methods for step acceptance roughly a filter method accepts a step that improves either the objective function value or the value of some function that measures the constraint violation the new algorithm does not compute or approximate any derivatives penalty constants or lagrange multipliers a key feature of the new algorithm is that it preserves the division into search and local poll steps which allows the explicit use of inexpensive surrogates or random search heuristics in the search step it is shown here that the algorithm identifies limit points at which optimality conditions depend on local smoothness of the functions and to a greater extent on the choice of a certain set of directions stronger optimality conditions are guaranteed for smoother functions and in the constrained case for a fortunate choice of the directions on which the algorithm depends these directional conditions generalize those given previously for linear constraints but they do not require a feasible starting point in the absence of general constraints the proposed algorithm and its convergence analysis generalize previous work on unconstrained bound constrained and linearly constrained generalized pattern search the algorithm is illustrated on some test examples and on an industrial wing planform engineering design application

efficient derivative pricing by the extended method of moments in this paper we introduce the extended method of moments xmm estimator this estimator accommodates a more general set of moment restrictions than the standard generalized method of moments gmm estimator more specifically the xmm differs from the gmm in that it can handle not only uniform conditional moment restrictions i e valid for any value of the conditioning variable but also local conditional moment restrictions valid for a given fixed value of the conditioning variable the local conditional moment restrictions are of special relevance in derivative pricing to reconstruct the pricing operator on a given day by using the information in a few cross sections of observed traded derivative prices and a time series of underlying asset returns the estimated derivative prices are consistent for a large time series dimension but a fixed number of cross sectionally observed derivative prices the asymptotic properties of the xmm estimator are nonstandard since the combination of uniform and local conditional moment restrictions induces different rates of convergence parametric and nonparametric for the parameters

a simple nonparametric approach to estimating the distribution of random coefficients in structural models abstract we explore least squares and likelihood nonparametric mixtures estimators of the joint distribution of random coefficients in structural models the estimators fix a grid of heterogeneous parameters and estimate only the weights on the grid points an approach that is computationally attractive compared to alternative nonparametric estimators we provide conditions under which the estimated distribution function converges to the true distribution in the weak topology on the space of distributions we verify most of the consistency conditions for three discrete choice models we also derive the convergence rates of the least squares nonparametric mixtures estimator under additional restrictions we perform a monte carlo study on a dynamic programming model

non parametric analysis of covariance an analysis of covariance model where the covariate effect is assumed only to be smooth is considered the possibility of different shapes of covariate effect in different groups is also allowed and tests of equality and of parallelism across groups are constructed these are implemented using gasser maller smoothing whose properties enable problems of bias to be avoided accurate moment based approximations are available for the distribution of each test statistic some data on spanish onions are used to contrast the non parametric approach with that of a nonlinear but parametric model a simulation study is also used to explore the properties of the non parametric tests

adaptive normalized quasi newton algorithms for extraction of generalized eigen pairs and their convergence analysis the main contributions of this paper are to propose and analyze fast and numerically stable adaptive algorithms for the generalized hermitian eigenvalue problem ghep which arises in many signal processing applications first for given explicit knowledge of a matrix pencil we formulate two novel deterministic discrete time ddt systems for estimating the generalized eigen pair eigenvector and eigenvalue corresponding to the largest smallest generalized eigenvalue by characterizing a generalized eigen pair as a stationary point of a certain function the proposed ddt systems can be interpreted as natural combinations of the normalization and quasi newton steps for finding the solution second we present adaptive algorithms corresponding to the proposed ddt systems moreover we establish rigorous analysis showing that for a step size within a certain range the sequence generated by the ddt systems converges to the orthogonal projection of the initial estimate onto the generalized eigensubspace corresponding to the largest smallest generalized eigenvalue numerical examples demonstrate the practical applicability and efficacy of the proposed adaptive algorithms

a filter method for solving nonlinear complementarity problems filter methods are extensively studied to handle nonlinear programming problems recently because of good numerical results filter techniques are attached importance to in this paper filter approaches are employed to tackle nonlinear complementarity problems ncps firstly ncp conditions are transformed into a nonlinear programming problem then to obtain a trial step the corresponding nonlinear programming problems are solved by some existing strategies moreover filter criterion is utilized to evaluate a trial iterate the purpose in this paper is to employ filter approaches to attack ncps in essence multi objective view is utilized to attack ncps because the idea of filter methods stems from multi objective problems furthermore a new filter method based on the special two objects which differs from others is brought forward moreover maratos effects are overcome in our new filter approach by weakening acceptable conditions

renormalizing upper and lower bounds for integrated risk in the white noise model where wt is brownian motion many results which might be difficult to prove in the density estimation or nonparametric regression context take on a more transparent form in this white noise model a sample size of n in the density estimation and nonparametric regression problems corresponds to o 0 o xn in 1 when ois suitably chosen in particular the tools of rescaling developed in low 1992 and donoho and low 1992 and the hardest one dimensional subfamily arguments of donoho and liu 1987 1991 have yielded a fairly complete picture of how to estimate both bounded and unbounded linear functionals on the basis of observations generated by 1 a separate literature is developing to show how to replace density estimation and regression problems by the corresponding white noise problems see for example low 1992 brown and low 1990 and donoho and low 1990 in this paper we focus attention on estimating the entire function f on the basis of the observation scheme given by 1 using integrated squared error as a measure of loss in particular we shall let r f o denote the minimax risk under this loss function that is

a trust region algorithm combining line search filter method with lagrange merit function for nonlinear constrained optimization this paper presents a trust region algorithm which employs line search filter technique with lagrange merit function for solving nonlinear equality constrained programming at the current iteration the general full trust region subproblem is decomposed into a pair of trust region subproblems in normal and tangential subspaces the trial step is given by solving these two trust region subproblems then different from traditional trust region method in which the next iterate is determined by the ratio of the actual reduction to the predicted reduction the step size is decided by interior backtracking line search together with filter method consequently the expensive computation raised by resolving trust region subproblem many times to determine a new iteration point in traditional trust region method can be reduced and the difficult decisions in regard to the choice of penalty parameters in the merit functions can be avoided by using filter technique the new method retains the global convergence to the first order critical points under some reasonable assumptions without using a merit function meanwhile by using lagrange function in the filter the method can overcome maratos effect without using second order correction step and hence the superlinear local convergence is achieved the preliminary numerical results are reported to show effectiveness of the proposed algorithm

convergence analysis of a deterministic discrete time system of feng s mca learning algorithm the convergence of minor component analysis mca algorithms is an important issue with bearing on the use of these methods in practical applications this correspondence studies the convergence of feng s mca learning algorithm via a corresponding deterministic discrete time ddt system some sufficient convergence conditions are obtained for feng s mca learning algorithm with constant learning rate simulations are carried out to illustrate the theory

local convergence of a trust region algorithm with line search filter technique for nonlinear constrained optimization a trust region algorithm in association with line search filter technique for solving nonlinear equality constrained programming is proposed in this paper in the current iteration the trial step providing sufficient descent is generated by solving a corresponding trust region subproblem then the step size is decided by backtracking line search together with filter technique to obtain the next iteration point the advantage of this method is that resolving trust region subproblem many times to determine a new iteration point in traditional trust region method can be avoided and hence the expensive computation can be lessened and the difficult decisions in regard to the choice of penalty parameters in the merit functions can be avoided by using filter technique second order correction steps are introduced in the proposed algorithm to overcome maratos effect convergence analysis shows that fast local convergence can be achieved under some mild assumptions the preliminary numerical results are reported

a trust region sqp filter method for nonlinear second order cone programming in this paper we consider the nonlinear second order cone programming problem by combining an sqp method and filter technique we present a trust region sqp filter method for solving this problem the proposed algorithm avoids using the classical merit function with penalty term furthermore under standard assumptions we prove that the iterative sequence generated by the presented algorithm converges globally preliminary numerical results indicate that the algorithm is promising

statistical estimation and optimal recovery new formulas are given for the minimax linear risk in estimating a linear functional of an unknown object from indirect data contaminated with random gaussian noise the formulas cover a variety of loss functions and do not require the symmetry of the convex a priori class it is shown that affine minimax rules are within a few percent of minimax even among nonlinear rules for a variety of loss functions it is also shown that difficulty of estimation is measured by the modulus of continuity of the functional to be estimated the method of proof exposes a correspondence between minimax affine estimates in the statistical estimation problem and optimal algorithms in the theory of optimal recovery

applications of second order cone programming in a second order cone program socp a linear function is minimized over the intersection of an affine set and the product of second order quadratic cones socps are nonlinear convex problems that include linear and convex quadratic programs as special cases but are less general than semidefinite programs sdps several efficient primaldual interior point methods for socp have been developed in the last few years after reviewing the basic theory of socps we describe general families of problems that tan be recast as socps these include robust linear programming and robust leastsquares problems problems involving sums or maxima of norms or with convex hyperbolic constraints we discuss a variety of engineering applications such as filter design antenna array weight design truss design and grasping forte optimization in robotics we describe an efficient primaldual interior point method for solving socps which shares many of the features of primaldual interior point methods for linear program

on the global convergence of a filter sqp algorithm a mechanism for proving global convergence in sqp filter methods for nonlinear programming nlp is described such methods are characterized by their use of the dominance concept of multiobjective optimization instead of a penalty parameter whose adjustment can be problematic the main point of interest is to demonstrate how convergence for nlp can be induced without forcing sufficient descent in a penalty type merit function the proof relates to a prototypical algorithm within which is allowed a range of specific algorithm choices associated with the hessian matrix representation updating the trust region radius and feasibility restoration

stochastic approach for the solution of multi pantograph differential equation arising in cell growth model in this paper a computational technique is introduced for the solution of the first order multi pantograph differential equation mpde through some well known optimization algorithms like sequential quadratic programming sqp and active set technique ast furthermore artificial neural network ann is used for networking of the first order multi pantograph differential equation in used to provide mathematical model based on unsupervised error for equation moreover mathematical modeling has been performed perfectly through multi runs for simulation to justify the better convergence of the solutions also two examples are presented to exhibit the aptitude of the method sqp and ast the comparative study will be made with reported techniques such as variational iteration technique vit 6 and collocation based on bernstein polynomial method bcm 6

weighted local regression and kernel methods for nonparametric curve fitting abstract weighted local regression a popular technique for smoothing scatterplots is shown to be asymptotically equivalent to certain kernel smoothers since both methods are local weighted averages of the data it is proved that in the fixed design regression model given a weighted local regression procedure with any weight function there is a corresponding kernel method such that the quotients of weights distributed by both methods tend uniformly to 1 as the number of observations increases to infinity it is demonstrated by examples that in some instances the weights are nearly the same for both methods even for small samples the asymptotic equivalence allows the derivation of the leading terms of the mean squared error and of the local limit distribution for weighted local regression further a close correspondence is found between the orders of the polynomial to be locally fitted in weighted local regression and the order number of vanishing moments of the kernel employed in the kernel smoot

design adaptive nonparametric regression abstract in this article we study the method of nonparametric regression based on a weighted local linear regression this method has advantages over other popular kernel methods moreover such a regression procedure has the ability of design adaptation it adapts to both random and fixed designs to both highly clustered and nearly uniform designs and even to both interior and boundary points it is shown that the local linear regression smoothers have high asymptotic efficiency i e can be 100 with a suitable choice of kernel and bandwidth among all possible linear smoothers including those produced by kernel orthogonal series and spline methods the finite sample property of the local linear regression smoother is illustrated via simulation studies nonparametric regression is frequently used to explore the association between covariates and responses there are many versions of kernel regression smoothers some estimators are not good for random designs such as in observational studies and

second order cone programming second order cone programming socp problems are convex optimization problems in which a linear function is minimized over the intersection of an affine linear manifold with the cartesian product of second order lorentz cones linear programs convex quadratic programs and quadratically constrained convex quadratic programs can all be formulated as socp problems as can many other problems that do not fall into these three categories these latter problems model applications from a broad range of fields from engineering control and finance to robust optimization and combinatorial optimization on the other hand semidefinite programming sdp that is the optimization problem over the intersection of an affine set and the cone of positive semidefinite matrices includes socp as a special case therefore socp falls between linear lp and quadratic qp programming and sdp like lp qp and sdp problems socp problems can be solved in polynomial time by interior point methods the computational effort per iteration required by these methods to solve socp problems is greater than that required to solve lp and qp problems but less than that required to solve sdp s of similar size and structure because the set of feasible solutions for an socp problem is not polyhedral as it is for lp and qp problems it is not readily apparent how to develop a simplex or simplex like method for socp while socp problems can be solved as sdp problems doing so is not advisable both on numerical grounds and computational complexity concerns for instance many of the problems presented in the survey paper of vandenberghe and boyd vb96 as examples of sdps can in fact be formulated as socps and should be solved as such in 2 3 below we give socp formulations for four of these examples the convex quadratically constrained quadratic programming qcqp problem problems involving fractional quadratic functions rutcor rutgers university e mail alizadeh rutcor rutgers edu research supported in part by the u s national science foundation grant ccr 9901991 ieor columbia university e mail gold ieor columbia edu research supported in part by the department of energy grant de fg02 92er25126 national science foundation grants dms 94 14438 cda 97 26385 and dms 01 04282

variance estimation in nonparametric regression via the difference sequence method consider a gaussian nonparametric regression problem having both an unknown mean function and unknown variance function this article presents a class of difference based kernel estimators for the variance function optimal convergence rates that are uniform over broad functional classes and bandwidths are fully characterized and asymptotic normality is also established we also show that for suitable asymptotic formulations our estimators achieve the minimax rate

local quasi likelihood with a parametric guide generalized linear models and the quasi likelihood method extend the ordinary regression models to accommodate more general conditional distributions of the response nonparametric methods need no explicit parametric specification and the resulting model is completely determined by the data themselves however nonparametric estimation schemes generally have a slower convergence rate such as the local polynomial smoothing estimation of nonparametric generalized linear models studied in fan heckman and wand j amer statist assoc 90 1995 141 150 in this work we propose a unified family of parametrically guided nonparametric estimation schemes this combines the merits of both parametric and nonparametric approaches and enables us to incorporate prior knowledge asymptotic results and numerical simulations demonstrate the improvement of our new estimation schemes over the original nonparametric counterpart

the triangular model with random coefficients abstract the triangular model is a very popular way to allow for causal inference in the presence of endogeneity in this model an outcome is determined by an endogenous regressor which in turn is first caused by an instrument we study the triangular model with random coefficients and additional exogenous regressors in both equations and establish non identification of the joint distribution of random coefficients this implies that counterfactual outcomes are not identified either non identification continues to hold if we confine ourselves to the joint distribution of coefficients in the outcome equation or indeed any marginal except the one on the endogenous regressor nonidentification prevails as well if we focus on means of random coefficients implying that iv is asymptotically biased based on these insights we derive bounds on the joint distribution of economically relevant functionals e g counterfactual outcomes and suggest an additional restriction that allows to point identify the distribution of random coefficients in the outcome equation we extend the model to cover the case where the regressors and instruments have limited support and analyze semi and nonparametric sample counterpart estimators in finite and large samples and we provide an application to consumer demand

a unifying approach to nonparametric regression estimation abstract there exists a variety of nonparametric regression estimators with cubic smoothing splines k nearest neighbor k nn estimators and various types of kernel estimators among the most popular a class of kernel estimators with local bandwidth depending on the density of the design points is introduced where the degree of design adaptation may be expressed by a single parameter in 0 1 the adaptation to the design is such that the bandwidth is made larger to a degree depending on where the design is thin special values of this parameter correspond approximately to the ordinary fixed width kernel estimator the smoothing spline and the k nn estimator hence this method offers a synthesis of some classical methods the same method allows estimation of derivatives by using appropriate kernels the influence of this degree of design adaptation on the integrated mean squared error is investigated there is no uniformly optimal solution the optimal solution depends in a complex way on de

convergence analysis of the ojan mca learning algorithm by the deterministic discrete time method minor component analysis mca is a statistical method of extracting the eigenvector associated with the smallest eigenvalue of the covariance matrix of input signals convergence is essential for mca algorithms towards practical applications traditionally the convergence of mca algorithms is indirectly analyzed via their corresponding deterministic continuous time dct systems however the dct method requires the learning rate to approach zero which is not reasonable in many applications due to the round off limitation and tracking requirements this paper studies the convergence of the deterministic discrete time ddt system associated with the ojan mca learning algorithm unlike the dct method the ddt method does not require the learning rate to approach zero in this paper some important convergence results are obtained for the ojan mca learning algorithm via the ddt method simulations are carried out to illustrate the theoretical results achieved

a smoothed em approach to indirect estimation problems with particular reference to stereology and emission tomography there are many practical problems where the observed data are not drawn directly from the density g of real interest but rather from another distribution derived from g by the application of an integral operator the estimation of g then entails both statistical and numerical difficulties a natural statistical approach is by maximum likelihood conveniently implemented using the em algorithm but this provides unsatisfactory reconstructions of g in this paper we modify the maximum likelihood em approach by introducing a simple smoothing step at each em iteration in our experience this algorithm converges in relatively few iterations to good estimates of g that do not depend on the choice of starting configuration some theoretical background is given that relates this smoothed em algorithm to a maximum penalized likelihood approach two applications are considered in detail the first is the classical stereology problem of determining particle size distributions from data collected on a plane section through a composite medium the second concerns the recovery of the structure of a section of the human body from external observations obtained by positron emission tomography for this problem we also suggest several technical improvements on existing methodology

testing for smooth structural changes in time series models via nonparametric regression checking parameter stability of econometric models is a long standing problem almost all existing structural change tests in econometrics are designed to detect abrupt breaks little attention has been paid to smooth structural changes which may be more realistic in economics we propose a consistent test for smooth structural changes as well as abrupt structural breaks with known or unknown change points the idea is to estimate smooth time varying parameters by local smoothing and compare the fitted values of the restricted constant parameter model and the unrestricted time varying parameter model the test is asymptotically pivotal and does not require prior information about the alternative a simulation study highlights the merits of the proposed test relative to a variety of popular tests for structural changes in an application we strongly reject the stability of univariate and multivariate stock return prediction models in the postwar and post oil shocks periods

convergence analysis of a deterministic discrete time system of oja s pca learning algorithm the convergence of oja s principal component analysis pca learning algorithms is a difficult topic for direct study and analysis traditionally the convergence of these algorithms is indirectly analyzed via certain deterministic continuous time dct systems such a method will require the learning rate to converge to zero which is not a reasonable requirement to impose in many practical applications recently deterministic discrete time ddt systems have been proposed instead to indirectly interpret the dynamics of the learning algorithms unlike dct systems ddt systems allow learning rates to be constant which can be a nonzero this paper will provide some important results relating to the convergence of a ddt system of oja s pca learning algorithm it has the following contributions 1 a number of invariant sets are obtained based on which we can show that any trajectory starting from a point in the invariant set will remain in the set forever thus the nondivergence of the trajectories is guaranteed 2 the convergence of the ddt system is analyzed rigorously it is proven in the paper that almost all trajectories of the system starting from points in an invariant set will converge exponentially to the unit eigenvector associated with the largest eigenvalue of the correlation matrix in addition exponential convergence rate are obtained providing useful guidelines for the selection of fast convergence learning rate 3 since the trajectories may diverge the careful choice of initial vectors is an important issue this paper suggests to use the domain of unit hyper sphere as initial vectors to guarantee convergence 4 simulation results will be furnished to illustrate the theoretical results achieved

total least squares approach for frequency estimation using linear prediction the resolution of the estimated closely spaced frequencies of the multiple sinusoids degrades as the signal to noise ratio snr of the received signal becomes low this resolution can be improved by using the total least squares tls method in solving the linear prediction lp equation this approach makes use of the singular value decomposition svd of the augmented matrix for low rank approximation to reduce the noise effect from both the observation vector and the lp data matrix simultaneously comparison is made to the principle eigenvector pe method of tufts and kumaresan both on theoretical and experimental grounds the tls algorithm exhibits superior performance over the pe method where low rank approximation is applied to the data matrix only

nonparametric maximum likelihood methods for binary response models with random coefficients the venerable method of maximum likelihood has found numerous recent applications in nonparametric estimation of regression and shape constrained densities for mixture models the nonparametric max

estimating semiparametric arch models by kernel smoothing methods we investigate a class of semiparametric arch models that includes as a special case the partially nonparametric pnp model introduced by engle and ng 1993 and which allows for both flexible dynamics and flexible function form with regard to the news impact function we show that the functional part of the model satisfies a type ii linear integral equation and give simple conditions under which there is a unique solution we propose an estimation method that is based on kernel smoothing and profiled likelihood we establish the distribution theory of the parametric components and the pointwise distribution of the nonparametric component of the model we also discuss efficiency of both the parametric part and the nonparametric part we investigate the performance of our procedures on simulated data and on a sample of s p500 index returns we find evidence of asymmetric news impact functions consistent with the parametric analysis

on nonparametric estimation of intercept and slope distributions in random coefficient regression an experiment records stimulus and response for a random sample of cases the relationship between response and stimulus is thought to be linear the values of the slope and intercept varying by case from such data we construct a consistent asymptotically normal nonparametric estimator for the joint density of the slope and intercept our methodology incorporates the radial projection slice theorem for the radon transform a technique for locally linear nonparametric regression and a tapered fourier inversion computationally the new density estimator is more feasible than competing nonparametric estimators one of which is based on moments and the other on minimum distance considerations

variability assessment in positron emission tomography and related generalized deconvolution models abstract the problem of variance assessment for positron emission tomography pet image reconstructions is considered in the context of generalized deconvolution a refinement of an approximate technique proposed by carson and colleagues is examined computational implications of representing the reconstruction kernel in terms of a weighted sum of gaussian densities are developed bias and variance characteristics of the resulting variance estimators are examined by numerical simulation for typical regions the error in estimated standard deviations is found to be on the order of 10 the use of smoothing to obtain more reliable pointwise variance estimators is described and some theoretical analysis of this technique is carried out for the pet application simulations suggest that the percent improvement in the root mean squared error accuracy of pointwise variance estimators obtained by smoothing can be on the order of 30 a practical application of the methodology to a pet study is presented

optimal inference in a class of regression models we consider the problem of constructing confidence intervals cis for a linear functional of a regression function such as its value at a point the regression discontinuity parameter or a regression coefficient in a linear or partly linear regression our main assumption is that the regression function is known to lie in a convex function class which covers most smoothness and or shape assumptions used in econometrics we derive finite sample optimal cis and sharp efficiency bounds under normal errors with known variance we show that these results translate to uniform over the function class asymptotic results when the error distribution is not known when the function class is centrosymmetric these efficiency bounds imply thatminimax cis are close to efficient at smooth regression functions this implies in particular that it is impossible to form cis that are tighter using data dependent tuning parameters and maintain coverage over the whole function class we specialize our results to inference in a linear regression and inference on the regression discontinuity parameter and illustrate them in an empirical application

a modified sqp filter method and its global convergence for current sequential programming sqp type algorithms there exist two problems one is that there are many quadratic programming subproblems are needed to be solved per iteration the other is that the search direction may be unbounded and caused the sequence to be divergent in this paper we presented a modified sqp filter method based on the modified quadratic subproblem proposed in zhou g l zhou a modified sqp method and its global convergence j global optim 11 1997 193 205 this method has no demand on the initial point and need not using a penalty parameter which could be problematic to obtain what is more the subproblem is feasible at each iterate point under some conditions the global convergence property is obtained

nonparametric errors in variables models with measurement errors on both sides of the equation abstract measurement errors are often correlated as in surveys where respondent s biases or tendencies to err affect multiple reported variables we extend schennach 2007 to identify moments of the conditional distribution of a true y given a true x when both are measured with error the measurement errors in y and x are correlated and the true unknown model of y given x has nonseparable model errors after showing nonparametric identification we provide a sieve generalized method of moments based estimator of the model and apply it to nonparametric engel curve estimation in our application measurement errors on the expenditures of a good y are by construction correlated with measurement errors in total expenditures x this problem which is present in many consumption data sets has been ignored in most demand applications we find that accounting for this problem casts doubt on hildenbrand s 1994 increasing dispersion assumption

nonparametric identification of the distribution of random coefficients in binary response static games of complete information this paper studies binary response static games of complete information allowing complex heterogeneity through a random coefficients specification the main result of the paper establishes nonparametric point identification of the joint density of all random coefficients except those on interaction effects under additional independence assumptions we identify the joint density of the interaction coefficients moreover we prove that in the presence of covariates that are common to both players the player specific coefficient densities are identified while the joint density of all random coefficients is not point identified however we do provide bounds on counterfactual probabilities that involve this joint density

on the optimal rates of convergence for nonparametric deconvolution problems deconvolution problems arise in a variety of situations in statistics an interesting problem is to estimate the density f of a random variable x based on n i i d observations from y x varepsilon where varepsilon is a measurement error with a known distribution in this paper the effect of errors in variables of nonparametric deconvolution is examined insights are gained by showing that the difficulty of deconvolution depends on the smoothness of error distributions the smoother the harder in fact there are two types of optimal rates of convergence according to whether the error distribution is ordinary smooth or supersmooth it is shown that optimal rates of convergence can be achieved by deconvolution kernel density estimators

minimax estimation of the mean of a normal distribution when the parameter space is restricted if x is a n theta 1 random variable let rho m be the minimax risk for estimation with quadratic loss subject to theta leq m then rho m 1 pi 2 m 2 o m 2 we exhibit estimates which are asymptotically minimax to this order as well as approximations to the least favorable prior distributions the approximate least favorable distributions correct to order m 2 have density m 1 cos 2 big frac pi 2m s big s leq m rather than the naively expected uniform density on lbrack m m rbrack we also show how our results extend to estimation of a vector mean and give some explicit solutions

the mca exin neuron for the minor component analysis the minor component analysis mca deals with the recovery of the eigenvector associated to the smallest eigenvalue of the autocorrelation matrix of the input data and is a very important tool for signal processing and data analysis it is almost exclusively solved by linear neurons this paper presents a linear neuron endowed with a novel learning law called mca exinn and analyzes its features the neural literature about mca is very poor in the sense that both a little theoretical basis is given almost always focusing on the ode asymptotic approximation and only experiments on toy problems at most four dimensional problems are presented without any numerical analysis this work addresses these problems and lays sound theoretical foundations for the neural mca theory in particular it classifies the mca neurons according to the riemannian metric and justifies from the analysis of the degeneracy of the error cost the different behavior in approaching convergence the cost landscape is studied and used as a basis for the analysis of the asymptotic behavior all the phases of the dynamics of the mca algorithms are investigated in detail and together with the numerical analysis lead to the identification of three possible kinds of divergence here called sudden dynamic and numerical the importance of the choice of low initial conditions is also explained a lot of importance is given to the experimental part where simulations on high dimensional problems are presented and analyzed the orthogonal regression or total least squares tls technique is also presented together with a real world application on the identification of the parameters of an electrical machine it can be concluded that mca exin is the best mca neuron in terms of stability no finite time divergence speed and accuracy

a kernel based parametric method for conditional density estimation a conditional density function which describes the relationship between response and explanatory variables plays an important role in many analysis problems in this paper we propose a new kernel based parametric method to estimate conditional density an exponential function is employed to approximate the unknown density and its parameters are computed from the given explanatory variable via a nonlinear mapping using kernel principal component analysis kpca we develop a new kernel function which is a variant to polynomial kernels to be used in kpca the proposed method is compared with the nadaraya watson estimator through numerical simulation and practical data experimental results show that the proposed method outperforms the nadaraya watson estimator in terms of revised mean integrated squared error rmise therefore the proposed method is an effective method for estimating the conditional densities

methods for estimating a conditional distribution function abstract motivated by the problem of setting prediction intervals in time series analysis we suggest two new methods for conditional distribution estimation the first method is based on locally fitting a logistic model and is in the spirit of recent work on locally parametric techniques in density estimation it produces distribution estimators that may be of arbitrarily high order but nevertheless always lie between 0 and 1 the second method involves an adjusted form of the nadaraya watson estimator it preserves the bias and variance properties of a class of second order estimators introduced by yu and jones but has the added advantage of always being a distribution itself our methods also have application outside the time series setting for example to quantile estimation for independent data this problem motivated the work of yu and jones

sharp adaptive estimation of the drift function for ergodic diffusions the global estimation problem of the drift function is considered for a large class of ergodic diffusion processes the unknown drift s is supposed to belong to a nonparametric class of smooth functions of order k 1 but the value of k is not known to the statistician a fully data driven procedure of estimating the drift function is proposed using the estimated risk minimization method the sharp adaptivity of this procedure is proven up to an optimal constant when the quality of the estimation is measured by the integrated squared error weighted by the square of the invariant density

a high dimensional wilks phenomenon a theorem by wilks asserts that in smooth parametric density estimation the difference between the maximum likelihood and the likelihood of the sampling distribution converges toward a chi square distribution where the number of degrees of freedom coincides with the model dimension this observation is at the core of some goodness of fit testing procedures and of some classical model selection methods this paper describes a non asymptotic version of the wilks phenomenon in bounded contrast optimization procedures using concentration inequalities for general functions of independent random variables it proves that in bounded contrast minimization as for example in statistical learning theory the difference between the empirical risk of the minimizer of the true risk in the model and the minimum of the empirical risk the excess empirical risk satisfies a bernstein like inequality where the variance term reflects the dimension of the model and the scale term reflects the noise conditions from a mathematical statistics viewpoint the significance of this result comes from the recent observation that when using model selection via penalization the excess empirical risk represents a minimum penalty if non asymptotic guarantees concerning prediction error are to be provided from the perspective of empirical process theory this paper describes a concentration inequality for the supremum of a bounded non centered actually non positive empirical process combining the now classical analysis of m estimation building on talagrand s inequality for suprema of empirical processes and versatile moment inequalities for functions of independent random variables this paper develops a genuine bernstein like inequality that seems beyond the reach of traditional tools

an effective bandwidth selector for local least squares regression abstract local least squares kernel regression provides an appealing solution to the nonparametric regression or scatterplot smoothing problem as demonstrated by fan for example the practical implementation of any scatterplot smoother is greatly enhanced by the availability of a reliable rule for automatic selection of the smoothing parameter in this article we apply the ideas of plug in bandwidth selection to develop strategies for choosing the smoothing parameter of local linear squares kernel estimators our results are applicable to odd degree local polynomial fits and can be extended to other settings such as derivative estimation and multiple nonparametric regression an implementation in the important case of local linear fits with univariate predictors is shown to perform well in practice a by product of our work is the development of a class of nonparametric variance estimators based on local least squares ideas and plug in rules for their implementation

second order cone programming relaxation of sensor network localization the sensor network localization problem has been much studied recently biswas and ye proposed a semidefinite programming sdp relaxation of this problem which has various nice properties and for which a number of solution methods have been proposed here we study a second order cone programming socp relaxation of this problem motivated by its simpler structure and its potential to be solved faster than sdp we show that the socp relaxation though weaker than the sdp relaxation has nice properties that make it useful as a problem preprocessor in particular sensors that are uniquely positioned among interior solutions of the socp relaxation are accurate up to the square root of the distance error thus these sensors which are easily identified are accurately positioned in our numerical simulation the interior solution found can accurately position up to 80 90p of the sensors we also propose a smoothing coordinate gradient descent method for finding an interior solution that is faster than an interior point method