conditional scores and optimal scores for generalized linear measurement error models abstract this paper studies estimation of the parameters of generalized linear models in canonical form when the explanatory vector is measured with independent normal error for the functional case i e when the explanatory vectors are fixed constants unbiased score functions are obtained by conditioning on certain sufficient statistics this work generalizes results obtained for logistic regression in the case that the explanatory vectors are independent and identically distributed with unknown distribtuion efficient score functions are obtained using the theory developed in begun et al 1983 keywords conditional score function efficient score function functional model generalized linear model measurement error structural model

adaptive estimation of the dynamics of a discrete time stochastic volatility model abstract this paper is concerned with the discrete time stochastic volatility model y i exp x i 2 i x i 1 b x i x i i 1 where only y i is observed the model is rewritten as a particular hidden model z i x i e i x i 1 b x i x i i 1 where i and e i are independent sequences of i i d noise moreover the sequences x i and e i are independent and the distribution of e is known then our aim is to estimate the functions b and 2 when only observations z 1 z n are available we propose to estimate b f and b 2 2 f and study the integrated mean square error of projection estimators of these functions on automatically selected projection spaces by ratio strategy estimators of b and 2 are then deduced the mean square risk of the resulting estimators are studied and their rates are discussed lastly simulation experiments are provided constants in the penalty functions defining the estimators are calibrated and the quality of the estimators is checked on several examples

finite time and fixed time input to state stability explicit and implicit approaches abstract the present article gathers the analysis of non asymptotic convergence rates finite time and fixed time with the property of input to state stability theoretical tools to determine this joint property are presented for the case where an explicit iss lyapunov function is known and when it remains in implicit form e g as a solution of an algebraic equation for the case of finite time input to state stability necessary and sufficient conditions are given whereas for the fixed time case only a sufficient condition is obtained academic examples and numerical simulations support the obtained results

discrete time homogeneity robustness and approximation in this paper we study robustness properties of discrete time homogeneous systems we also analyse stability and robustness properties of a more general class of nonlin ear discrete time systems that can be approximated by homogeneous ones we apply the results to the investigation of stability properties of discretized continuous time systems

nonlinear feedback design for fixed time stabilization of linear control systems two types of nonlinear control algorithms are presented for uncertain linear plants controllers of the first type are stabilizing polynomial feedbacks that allow to adjust a guaranteed convergence time of system trajectories into a prespecified neighborhood of the origin independently on initial conditions the control design procedure uses block control principles and finite time attractivity properties of polynomial feedbacks controllers of the second type are modifications of the second order sliding mode control algorithms they provide global finite time stability of the closed loop system and allow to adjust a guaranteed settling time independently on initial conditions control algorithms are presented for both single input and multi input systems theoretical results are supported by numerical simulations

optimal rates of convergence for deconvolving a density abstract suppose that the sum of two independent random variables x and z is observed where z denotes measurement error and has a known distribution and where the unknown density f of x is to be estimated one application is the estimation of a prior density for a sequence of location parameters a second application arises in the errors in variables problem for nonlinear and generalized linear models when one attempts to model the distribution of the true but unobservable covariates this article shows that if z is normally distributed and f has k bounded derivatives then the fastest attainable convergence rate of any nonparametric estimator of f is only log n k 2 therefore deconvolution with normal errors may not be a practical proposition other error distributions are also treated stefanski carroll 1987a estimators achieve the optimal rates the results given have versions for multiplicative errors where they imply that even optimal rates are exceptionally slow

on errors in variables for binary regression models summary we consider binary regression models when some of the predictors are measured with error for normal measurement errors structural maximum likelihood estimates are considered we show that if the measurement error is large the usual estimate of the probability of the event in question can be substantially in error especially for high risk groups in the situation of large measurement error we investigate a conditional maximum likelihood estimator and its properties

efficient estimation in the errors in variables model we consider efficient estimation of the slope in the errors in variables model with normal error when either the ratio of error variances is known and the distribution of the independent is arbitrary and unknown or the distribution of the independent variable is not gaussian or degenerate we calculate information bounds and exhibit estimates achieving these bounds using an initial minimum distance estimate and suitable estimates of the efficient score function

a multi manifold discriminant analysis method for image feature extraction in this paper we propose a multi manifold discriminant analysis mmda method for an image feature extraction and pattern recognition based on graph embedded learning and under the fisher discriminant analysis framework in an mmda the within class graph and between class graph are respectively designed to characterize the within class compactness and the between class separability seeking for the discriminant matrix to simultaneously maximize the between class scatter and minimize the within class scatter in addition in an mmda the within class graph can represent the sub manifold information while the between class graph can represent the multi manifold information the proposed mmda is extensively examined by using the feret ar and orl face databases and the polyu finger knuckle print databases the experimental results demonstrate that an mmda is effective in feature extraction leading to promising image recognition performance

learning linear discriminant projections for dimensionality reduction of image descriptors in this paper we present linear discriminant projections ldp for reducing dimensionality and improving discriminability of local image descriptors we place ldp into the context of state of the art discriminant projections and analyze its properties ldp requires a large set of training data with point to point correspondence ground truth we demonstrate that training data produced by a simulation of image transformations leads to nearly the same results as the real data with correspondence ground truth this makes it possible to apply ldp as well as other discriminant projection approaches to the problems where the correspondence ground truth is not available such as image categorization we perform an extensive experimental evaluation on standard data sets in the context of image matching and categorization we demonstrate that ldp enables significant dimensionality reduction of local descriptors and performance increases in different applications the results improve upon the state of the art recognition performance with simultaneous dimensionality reduction from 128 to 30

regressions with berkson errors in covariates a nonparametric approach this paper establishes that so called instrumental variables enable the identification and the estimation of a fully nonparametric regression model with berkson type measurement error in the regressors an estimator is proposed and proven to be consistent its practical performance and feasibility are investigated via monte carlo simulations as well as through an epidemiological application investigating the effect of particulate air pollution on respiratory health these examples illustrate that berkson errors can clearly not be neglected in nonlinear regression models and that the proposed method represents an effective remedy

deconvolution density estimation on so n this paper develops nonparametric deconvolution density estimation over so n the group of n x n orthogonal matrices of determinant 1 the methodology is to use the group and manifold structures to adapt the euclidean deconvolution techniques to this lie group environment this is achieved by employing the theory of group representations explicit to so n general consistency results are obtained with specific rates of convergence achieved under sufficient smoothness conditions application to empirical bayes prior estimation and inference is also discussed

methodology for non parametric deconvolution when the error distribution is unknown summary in the non parametric deconvolution problem to estimate consistently a density or distribution from a sample of data contaminated by additive random noise it is often assumed that the noise distribution is completely known or that an additional sample of replicated or validation data is available methods also have been suggested for estimating the scale of the error distribution but they require somewhat restrictive smoothness assumptions on the signal distribution which can be difficult to verify in practice we take a completely new approach to the problem not requiring extra data of any type we argue that data rarely come from a simple regular distribution and that this can be exploited to estimate the signal distributions by using a simple procedure our method can be extended to other problems involving errors in variables such as non parametric regression estimation its performance in practice is remarkably good often equalling even unexpectedly the performance of techniques that use additional data to estimate the unknown error distribution

dimensionality reduction by mixed kernel canonical correlation analysis in this paper we propose a novel method named mixed kernel cca mkcca to achieve easy yet accurate implementation of dimensionality reduction mkcca consists of two major steps first the high dimensional data space is mapped into the reproducing kernel hilbert space rkhs rather than the hilbert space with a mixture of kernels i e a linear combination between a local kernel and a global kernel meanwhile a uniform design for experiments with mixtures is also introduced for model selection second in the new rkhs kernel cca is further improved by performing principal component analysis pca followed by cca for effective dimensionality reduction we prove that mkcca can actually be decomposed into two separate components i e pca and cca which can be used to better remove noises and tackle the issue of trivial learning existing in cca or traditional kernel cca after this the proposed mkcca can be implemented in multiple types of learning such as multi view learning supervised learning semi supervised learning and transfer learning with the reduced data we show its superiority over existing methods in different types of learning by extensive experimental results

homogeneous approximation recursive observer design and output feedback we introduce two new tools that can be useful in nonlinear observer and output feedback design the first one is a simple extension of the notion of homogeneous approximation to make it valid both at the origin and at infinity homogeneity in the bi limit exploiting this extension we give several results concerning stability and robustness for a homogeneous in the bi limit vector field the second tool is a new recursive observer design procedure for a chain of integrator combining these two tools we propose a new global asymptotic stabilization result by output feedback for feedback and feedforward systems

a semiparametric method for estimating nonlinear autoregressive model with dependent errors abstract the first order nonlinear autoregressive model is considered and a semiparametric method is proposed to estimate regression function in the presented model dependent errors are defined as first order autoregressive ar 1 the conditional least squares method is used for parametric estimation and the nonparametric kernel approach is applied to estimate regression adjustment in this case some asymptotic behaviors and simulated results for the semiparametric method are presented furthermore the method is applied for the financial data in iran s tejarat bank

sampling methods for the nystrom method the nystrom method is an efficient technique to generate low rank matrix approximations and is used in several large scale learning applications a key aspect of this method is the procedure according to which columns are sampled from the original matrix in this work we explore the efficacy of a variety of fixed and adaptive sampling schemes we also propose a family of ensemble based sampling algorithms for the nystrom method we report results of extensive experiments that provide a detailed comparison of various fixed and adaptive sampling techniques and demonstrate the performance improvement associated with the ensemble nystrom method when used in conjunction with either fixed or adaptive sampling schemes corroborating these empirical findings we present a theoretical analysis of the nystrom method providing novel error bounds guaranteeing a better convergence rate of the ensemble nystrom method in comparison to the standard nystrom method

implicit euler numerical scheme and chattering free implementation of sliding mode systems in this paper it is shown that the implicit euler time discretization of some classes of switching systems with sliding modes yields a very good stabilization of the trajectory and of its derivative on the sliding surface therefore the spurious oscillations which are pointed out elsewhere when an explicit method is used are avoided moreover the method an event capturing or time stepping algorithm allows for multiple switching surfaces i e a sliding surface of codimension 2 the details of the implementation are given and numerical examples illustrate the developments this method may be an alternative method for chattering suppression keeping the intrinsic discontinuous nature of the dynamics on the sliding surfaces links with discrete time sliding mode controllers are studied

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

homogeneous lyapunov function for homogeneous continuous vector field the goal of this article is to provide a construction of a homogeneous lyapunov function p associated with a system of differential equations j f x x r n 1 under the hypotheses 1 f c r n vanishes at x 0 and is homogeneous 2 the zero solution of this system is locally asymptotically stable moreover the lyapunov function v x tends to infinity with 1 x i and belongs to c r 0 r n cp with p e as large as wanted as application to the theory of homogeneous systems we present two well known results of robustness in a slightly extended form and with simpler proofs

verification of iss iiss and ioss properties applying weighted homogeneity several conditions are proposed to check different robustness properties iss iiss ioss and oss for generic nonlinear systems applying the weighted homogeneity concept global or local the advantages of this result are that under some mild conditions the system robustness can be established as a function of the degree of homogeneity

riemannian manifold learning recently manifold learning has been widely exploited in pattern recognition data analysis and machine learning this paper presents a novel framework called riemannian manifold learning rml based on the assumption that the input high dimensional data lie on an intrinsically low dimensional riemannian manifold the main idea is to formulate the dimensionality reduction problem as a classical problem in riemannian geometry that is how to construct coordinate charts for a given riemannian manifold we implement the riemannian normal coordinate chart which has been the most widely used in riemannian geometry for a set of unorganized data points first two input parameters the neighborhood size k and the intrinsic dimension d are estimated based on an efficient simplicial reconstruction of the underlying manifold then the normal coordinates are computed to map the input high dimensional data into a low dimensional space experiments on synthetic data as well as real world images demonstrate that our algorithm can learn intrinsic geometric structures of the data preserve radial geodesic distances and yield regular embeddings

on deconvolution with repeated measurements in a large class of statistical inverse problems it is necessary to suppose that the transformation that is inverted is known although in many applications it is unrealistic to make this assumption the problem is often insoluble without it however if additional data are available then it is possible to estimate consistently the unknown error density data are seldom available directly on the transformation but repeated or replicated measurements increasingly are becoming available such data consist of intrinsic values that are measured several times with errors that are generally independent working in this setting we treat the nonparametric deconvolution problems of density estimation with observation errors and regression with errors in variables we show that even if the number of repeated measurements is quite small it is possible for modified kernel estimators to achieve the same level of performance they would if the error distribution were known indeed density and regression estimators can be constructed from replicated data so that they have the same first order properties as conventional estimators in the known error case without any replication but with sample size equal to the sum of the numbers of replicates practical methods for constructing estimators with these properties are suggested involving empirical rules for smoothing parameter choice

trace ratio problem revisited dimensionality reduction is an important issue in many machine learning and pattern recognition applications and the trace ratio tr problem is an optimization problem involved in many dimensionality reduction algorithms conventionally the solution is approximated via generalized eigenvalue decomposition due to the difficulty of the original problem however prior works have indicated that it is more reasonable to solve it directly than via the conventional way in this brief we propose a theoretical overview of the global optimum solution to the tr problem via the equivalent trace difference problem eigenvalue perturbation theory is introduced to derive an efficient algorithm based on the newton raphson method theoretical issues on the convergence and efficiency of our algorithm compared with prior literature are proposed and are further supported by extensive empirical results

backward euler discretization of fully nonlinear parabolic problems this paper is concerned with the time discretization of nonlinear evolution equations we work in an abstract banach space setting of analytic semigroups that covers fully nonlinear parabolic initial boundary value problems with smooth coefficients we prove convergence of variable stepsize backward euler discretizations under various smoothness assumptions on the exact solution we further show that the geometric properties near a hyperbolic equilibrium are well captured by the discretization a numerical example is given

discretization of homogeneous systems using euler method with a state dependent step numeric approximations to the solutions of asymptotically stable homogeneous systems by euler method with a step of discretization scaled by the state norm are investigated for the explicit and implicit integration schemes it is proven that for a sufficiently small discretization step the convergence of the approximating solutions to zero can be guaranteed globally in a finite or a fixed time depending on the degree of homogeneity of the system but in an infinite number of discretization iterations the maximal admissible step can be estimated by analyzing the system properties on the sphere it is shown that the absolute and relative errors of the discretizations are globally bounded functions thus the approximations approaching the solutions with the step converging to zero in addition it is established that the proposed discretization approach preserves robustness with respect to exogenous perturbations efficiency of the designed discretization algorithms is demonstrated in simulations

discriminant neighborhood embedding for classification in this paper a novel subspace learning method called discriminant neighborhood embedding dne is proposed for pattern classification we suppose that multi class data points in high dimensional space tend to move due to local intra class attraction or inter class repulsion and the optimal embedding from the point of view of classification is discovered consequently after being embedded into a low dimensional subspace data points in the same class form compact submanifod whereas the gaps between submanifolds corresponding to different classes become wider than before experiments on the umist and mnist databases demonstrate the effectiveness of our method

discrete time equivalents of the super twisting algorithm abstract in this paper entirely new discrete time variants of the super twisting algorithm are presented the development of these discrete time equivalents is based on certain criteria e g the approximation of the continuous time super twisting algorithm as the discretization time tends to zero in contrast to the classical explicit euler discretized super twisting dynamics the proposed schemes are exact in the sense that in the unperturbed case the controllers ensure local convergence to the origin oscillations of the system states caused by the discrete time implementation of the super twisting algorithm are avoided the superiority of the developed control laws is demonstrated in simulation examples as well as in a real world application these examples reveal that the standard accuracy of the homogeneous second order sliding mode is preserved and in contrast to the explicit euler discretized algorithm in the presence of exact discrete measurements the precision of the controlled variable is insensitive to oversized control gains

homogeneity approach to high order sliding mode design it is shown that a general uncertain single input single output regulation problem is solvable only by means of discontinuous control laws giving rise to the so called high order sliding modes the homogeneity properties of the corresponding controllers yield a number of practically important features in particular finite time convergence is proved and asymptotic accuracy is calculated in a very general way in the presence of input noises discrete measurements and switching delays a robust homogeneous differentiator is included in the control structure thus yielding robust output feedback controllers with finite time convergence it is demonstrated that homogeneity features significantly simplify the design and investigation of a new family of high order sliding mode controllers finally simulation results are presented

convolution without independence abstract widely used convolution and deconvolution techniques traditionally rely on independence assumptions often criticized as being strong we observe that the convolution theorem actually holds under a weaker assumption known as subindependence we show that this notion is arguably as weak as a conditional mean assumption we report various simple characterizations of subindependence and devise constructive methods to generate subindependent random variables we extend subindependence to multivariate settings and propose the new concepts of conditional and mean subindependence relevant to measurement error problems we finally introduce three tests of subindependence based on characteristic functions generalized method of moments and randomization respectively

multi manifold lle learning in pattern recognition this paper introduces multiple manifold locally linear embedding mm lle learning this method learns multiple manifolds corresponding to multiple classes in a data set the proposed approach to manifold learning includes a supervised form of neighborhood selection in learning individual manifolds that correspond to each class of data furthermore mm lle uses manifold manifold distance mmd as a measure to find the optimum low dimensional space needed to achieve high classification accuracy when classifying new data samples in addition to the conventional classification techniques used in the past literature to classify new data in the manifold space we introduce a point to manifold distance pmd metric used to measure the distance between points and manifolds experimental results reported in this paper compare the recognition rates for a number of different manifold learning methods the proposed mm lle technique has various applications in classification and object recognition highlightswe introduce multiple manifold locally linear embedding mm lle learning this method learns multiple manifolds corresponding to multiple data classes mm lle uses supervised neighborhood selection in learning multiple manifolds it finds the optimum low dimensional space by minimizing the nearness of manifolds results show that mm lle outperforms many well known manifold learning algorithms

accelerating t sne using tree based algorithms the paper investigates the acceleration of t sne an embedding technique that is commonly used for the visualization of high dimensional data in scatter plots using two tree based algorithms in particular the paper develops variants of the barnes hut algorithm and of the dual tree algorithm that approximate the gradient used for learning t sne embeddings in o n log n our experiments show that the resulting algorithms substantially accelerate t sne and that they make it possible to learn embeddings of data sets with millions of objects somewhat counterintuitively the barnes hut variant of t sne appears to outperform the dual tree variant

low order approximations in deconvolution and regression with errors in variables summary we suggest two new methods which are applicable to both deconvolution and regression with errors in explanatory variables for nonparametric inference the two approaches involve kernel or orthogonal series methods they are based on defining a low order approximation to the problem at hand and proceed by constructing relatively accurate estimators of that quantity rather than attempting to estimate the true target functions consistently of course both techniques could be employed to construct consistent estimators but in many contexts of importance e g those where the errors are gaussian consistency is from a practical viewpoint an unattainable goal we rephrase the problem in a form where an explicit interpretable low order approximation is available the information that we require about the error distribution the error in variables distribution in the case of regression is only in the form of low order moments and so is readily obtainable by a rudimentary analysis of indirect measurements of errors e g through repeated measurements in particular we do not need to estimate a function such as a characteristic function which expresses detailed properties of the error distribution this feature of our methods coupled with the fact that all our estimators are explicitly defined in terms of readily computable averages means that the methods are particularly economical in computing time

instrumental variable estimation of nonlinear errors in variables models this paper establishes that instruments enable the identification of nonparametric regression models in the presence of measurement error by providing a closed form solution for the regression function in terms of fourier transforms of conditional expectations of observable variables for parametrically specified regression functions we propose a root n consistent and asymptotically normal estimator that takes the familiar form of a generalized method of moments estimator with a plugged in nonparametric kernel density estimate both the identification and the estimation methodologies rely on fourier analysis and on the theory of generalized functions the finite sample properties of the estimator are investigated through monte carlo simulations

color perception of diffusion tensor images using hierarchical manifold learning abstract the color perception of diffusion tensor images dti by using voxel based statistical analysis suffers from high computational cost and vague regional structure to address these issues we therefore propose a novel approach for color perception of dti based on hierarchical manifold learning first the selection of the representative nodes as seeds within similar region to build them into the bottom to up hierarchical structure is derived from the algebraic multigrid and multi scale graph partitioning next the low dimensional coordinates of the top layer seeds are calculated using manifold based techniques with a new distance metric and mapping of these coordinates into the rgb color space last the color perception of dti is obtained through interpolating the seeds to the bottom layer of all nodes the experimental results demonstrate that the proposed algorithm can reduce the computation complexity from o n 3 based on algorithms in the literature ghassan et al 2011 9 to o n 2 and highlight the different regional structures of the brain via color perception of variation

density estimation in the presence of heteroscedastic measurement error of unknown type using phase function deconvolution it is important to properly correct for measurement error when estimating density functions associated with biomedical variables these estimators that adjust for measurement error are broadly referred to as density deconvolution estimators while most methods in the literature assume the distribution of the measurement error to be fully known a recently proposed method based on the empirical phase function epf can deal with the situation when the measurement error distribution is unknown the epf density estimator has only been considered in the context of additive and homoscedastic measurement error however the measurement error of many biomedical variables is heteroscedastic in nature in this paper we developed a phase function approach for density deconvolution when the measurement error has unknown distribution and is heteroscedastic a weighted epf wepf is proposed where the weights are used to adjust for heteroscedasticity of measurement error the asymptotic properties of the wepf estimator are evaluated simulation results show that the weighting can result in large decreases in mean integrated squared error when estimating the phase function the estimation of the weights from replicate observations is also discussed finally the construction of a deconvolution density estimator using the wepf is compared with an existing deconvolution estimator that adjusts for heteroscedasticity but assumes the measurement error distribution to be fully known the wepf estimator proves to be competitive especially when considering that it relies on minimal assumption of the distribution of measurement error

multivalued robust tracking control of lagrange systems continuous and discrete time algorithms the robust trajectory tracking of fully actuated lagrange systems is studied exogenous perturbations as well as parameter uncertainties are taken into account a family of set valued passivity based controllers is proposed including first order sliding mode schemes the existence of solutions and the stability of the closed loop system are established in continuous time an implicit discretization approach is proposed and the well posedness and the stability of the closed loop system are studied numerical simulations illustrate the effectiveness of the proposed discrete time controller

nonlinear dimensionality reduction by topologically constrained isometric embedding many manifold learning procedures try to embed a given feature data into a flat space of low dimensionality while preserving as much as possible the metric in the natural feature space the embedding process usually relies on distances between neighboring features mainly since distances between features that are far apart from each other often provide an unreliable estimation of the true distance on the feature manifold due to its non convexity distortions resulting from using long geodesics indiscriminately lead to a known limitation of the isomap algorithm when used to map non convex manifolds presented is a framework for nonlinear dimensionality reduction that uses both local and global distances in order to learn the intrinsic geometry of flat manifolds with boundaries the resulting algorithm filters out potentially problematic distances between distant feature points based on the properties of the geodesics connecting those points and their relative distance to the boundary of the feature manifold thus avoiding an inherent limitation of the isomap algorithm since the proposed algorithm matches non local structures it is robust to strong noise we show experimental results demonstrating the advantages of the proposed approach over conventional dimensionality reduction techniques both global and local in nature

binary regression using an extended beta binomial distribution with discussion of correlation induced by covariate measurement errors abstract the beta binomial distribution is extended to allow negative correlations among binary variates within an experimental unit regression models are proposed for both the binary variate response rate and for the pairwise correlation between binary variates and corresponding likelihood estimation procedures are described binary regression problems are also considered in which measurement errors in the regression variables represent the sole source of overdispersion some corresponding response rate estimation procedures are described and illustrated

local indirect least squares and average marginal effects in nonseparable structural systems we study the scope of local indirect least squares lils methods for nonparametrically estimating average marginal effects of an endogenous cause x on a response y in triangular structural systems that need not exhibit linearity separability or monotonicity in scalar unobservables one main finding is negative in the fully nonseparable case lils methods cannot recover the average marginal effect lils methods can nevertheless test the hypothesis of no effect in the general nonseparable case we provide new nonparametric asymptotic theory treating both the traditional case of observed exogenous instruments z and the case where one observes only error laden proxies for z

supervised optimal locality preserving projection in the past few years the computer vision and pattern recognition community has witnessed a rapid growth of a new kind of feature extraction method the manifold learning methods which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure among these methods locality preserving projection lpp is one of the most promising feature extraction techniques unlike the unsupervised learning scheme of lpp this paper follows the supervised learning scheme i e it uses both local information and class information to model the similarity of the data based on novel similarity we propose two feature extraction algorithms supervised optimal locality preserving projection solpp and normalized laplacian based supervised optimal locality preserving projection nl solpp optimal here means that the extracted features via solpp or nl solpp are statistically uncorrelated and orthogonal we compare the proposed solpp and nl solpp with lpp orthogonal locality preserving projection olpp and uncorrelated locality preserving projection ulpp on publicly available data sets experimental results show that the proposed solpp and nl solpp achieve much higher recognition accuracy

distance metric learning with eigenvalue optimization the main theme of this paper is to develop a novel eigenvalue optimization framework for learning a mahalanobis metric within this context we introduce a novel metric learning approach called dml eig which is shown to be equivalent to a well known eigenvalue optimization problem called minimizing the maximal eigenvalue of a symmetric matrix overton 1988 lewis and overton 1996 moreover we formulate lmnn weinberger et al 2005 one of the state of the art metric learning methods as a similar eigenvalue optimization problem this novel framework not only provides new insights into metric learning but also opens new avenues to the design of efficient metric learning algorithms indeed first order algorithms are developed for dml eig and lmnn which only need the computation of the largest eigenvector of a matrix per iteration their convergence characteristics are rigorously established various experiments on benchmark data sets show the competitive performance of our new approaches in addition we report an encouraging result on a difficult and challenging face verification data set called labeled faces in the wild lfw

predictive inference for integrated volatility numerous volatility based derivative products have been engineered in recent years this has led to interest in constructing conditional predictive densities and confidence intervals for integrated volatility in this article we propose nonparametric estimators of the aforementioned quantities based on model free volatility estimators we establish consistency and asymptotic normality for the feasible estimators and study their finite sample properties through a monte carlo experiment finally using data from the new york stock exchange we provide an empirical application to volatility directional predictability

discriminative multimanifold analysis for face recognition from a single training sample per person conventional appearance based face recognition methods usually assume that there are multiple samples per person mspp available for discriminative feature extraction during the training phase in many practical face recognition applications such as law enhancement e passport and id card identification this assumption however may not hold as there is only a single sample per person sspp enrolled or recorded in these systems many popular face recognition methods fail to work well in this scenario because there are not enough samples for discriminant learning to address this problem we propose in this paper a novel discriminative multimanifold analysis dmma method by learning discriminative features from image patches first we partition each enrolled face image into several nonoverlapping patches to form an image set for each sample per person then we formulate the sspp face recognition as a manifold manifold matching problem and learn multiple dmma feature spaces to maximize the manifold margins of different persons finally we present a reconstruction based manifold manifold distance to identify the unlabeled subjects experimental results on three widely used face databases are presented to demonstrate the efficacy of the proposed approach

estimating linear statistical relationships this paper on estimating linear statistical relationships includes three lectures on linear functional and structural relationships factor analysis and simultaneous equations models the emphasis is on relating the several models by a general approach and on the similarity of maximum likelihood estimators under normality in the different models in the first two lectures the observable vector is decomposed into a systematic part and a random error the systematic part satisfies the linear relationships estimators are derived for several cases and some of their properties given estimation of the coefficients of a single equation in a simultaneous equations model is shown to be equivalent to estimation of linear functional relationships

discrete transform approach to deconvolution problems summary if fourier series are used as the basis for inference in deconvolution problems the effects of the errors factorise out in a way that is easily exploited empirically this property is the consequence of elementary addition formulae for sine and cosine functions and is not readily available when one is using methods based on other orthogonal series or on continuous fourier transforms it allows relatively simple estimators to be constructed founded on the addition of finite series rather than on integration the performance of these methods can be particularly effective when edge effects are involved since cosine series estimators are quite resistant to boundary problems in this context we point to the advantages of trigonometric series methods for density deconvolution they have better mean squared error performance when edge effects are involved they are particularly easy to code and they admit a simple approach to empirical choice of smoothing parameter in which a version of thresholding familiar in wavelet based inference is used in place of conventional smoothing applications to other deconvolution problems are briefly discussed

multi manifold discriminant isomap for visualization and classification isomap aims to recover the intrinsic geometric structure of manifold by preserving geodesic distances between all pairs of data points however it is an unsupervised dimensionality reduction method usually using class label information can increase the discriminating capability hence a new supervised isomap is proposed in this paper dubbed multi manifold discriminant isomap mmd isomap first data points are partitioned into different manifolds according to their class label information then mmd isomap aims at seeking an optimal nonlinear subspace to preserve the geometrical structure of each manifold according to the isomap criterion meanwhile to enhance the discriminating capability by maximizing the distances between data points of different manifolds finally the corresponding optimization problem is solved by using a majorization algorithm furthermore two new numerical metrics are designed to measure the performance of dimensionality reduction method in both visualization and classification experiments mmd isomap achieves improved performance over many state of the art methods preserve the geometrical structure and enhance the discriminating capability smacof algorithm is introduced to solve the optimization problem two metrics are designed to evaluate dimensionality reduction methods experiments show the effectiveness of the method in visualization and classification

realization and discretization of asymptotically stable homogeneous systems sufficient conditions for the existence and convergence to zero of numeric approximations to solutions of asymptotically stable homogeneous systems are obtained for the explicit and implicit euler integration schemes it is shown that the explicit euler method has certain drawbacks for the global approximation of homogeneous systems with nonzero degrees whereas the implicit euler scheme ensures convergence of the approximating solutions to zero properties of absolute and relative errors of the respective discretizations are investigated

granger causality driven ahp for feature weighted knn the knn algorithm remains a popular choice for pattern classification till date due to its non parametric nature easy implementation and the fact that its classification error is bounded by twice the bayes error in this paper we show that the performance of the knn classifier improves significantly from the use of training class wise group statistics based two criteria during pairwise comparison of features in a given dataset granger causality is employed to assign preferences to each criteria analytic hierarchy process ahp is applied to obtain weights for different features from the two criteria and their preferences finally these weights are used to build a weighted distance function for the knn classification comprehensive experimentation on fifteen benchmark datasets of the uci machine learning repository clearly reveals the supremacy of the proposed granger causality driven ahp induced knn algorithm over the knn method with many different distance metrics and with various feature selection strategies in addition the proposed method is also shown to perform well on high dimensional face and hand writing recognition datasets highlightsfeature weighting for knn by a multi criteria based decision analysis tool called ahp automated weight assignment in criteria matrix of ahp using group statistics criteria preference selection in ahp with granger causality superior classification performance over knn with many other feature weighting selection methods

lyapunov stability and performance analysis of the implicit discrete sliding mode control discrete time sliding mode controllers with an implicit discretization of the signum function are considered with a proper choice of the equivalent part of the control the resulting controller is shown to be lyapunov stable with finite time convergence of the sliding variable to 0 the convergence of the control input as the sampling period goes to 0 to the continuous time one is shown the robustness with respect to matching perturbations is also investigated the discretization performance in terms of the error order is studied for different discretizations of the equivalent part of the input numerical and experimental results illustrate and support the analysis

covariate measurement error in generalized linear models summary the em algorithm is used to obtain estimators of regression coefficients for generalized linear models with canonical link when normally distributed covariates are masked by normally distributed measurement errors by casting the true covariates as missing data the em procedure suggests an iterative scheme in which each cycle consists of an e step requiring the computation of approximate first and second conditional moments of the true covariates given the observed data followed by an m step in which regression parameters are updated by iteratively reweighted least squares based on these approximations the proposed procedure is compared numerically with the exact maximum likelihood solution obtained by using gaussian quadrature instead of the approximations in the e step of the em algorithm and with alternative estimators for simple logistic regression with measurement error the results for the proposed procedure are encouraging a procedure is proposed in this paper for estimating regression coefficients in generalized linear models when one or more of the covariates is measured with error this work is related to the paper by carroll et al 1984 in which maximum likelihood estimates are considered for the structural logistic regression model these authors provide estimates for the computationally simpler probit regression model and suggest that the structural logistic regression can be done in principle this paper follows up on that suggestion but differs in detail from similar work by armstrong 1985 other procedures with related aims have been suggested by stefanski carroll 1985 wolter fuller 1982 and prentice 1982 for a quick introduction to the proposed method it is convenient to display the naive estimator which ignores measurement error in its standard computational form if yi is an observed response variable and xi is the observed covariate then for the model which equates the canonical parameter of the distribution of yi to xtf3 the maximum likelihood estimator of f3 can be obtained by iteratively reweighted least squares the estimate of f3 after s 1 cycles is given by

minimum distance regression model checking with berkson measurement errors lack of fit testing of a regression model with berkson measurement error has not been discussed in the literature to date to fill this void we propose a class of tests based on minimized integrated square distances between a nonparametric regression function estimator and the parametric model being fitted we prove asymptotic normality of these test statistics under the null hypothesis and that of the corresponding minimum distance estimators under minimal conditions on the model being fitted we also prove consistency of the proposed tests against a class of fixed alternatives and obtain their asymptotic power against a class of local alternatives orthogonal to the null hypothesis these latter results are new even when there is no measurement error a simulation that is included shows very desirable finite sample behavior of the proposed inference procedures

regression analysis with covariates that have heteroscedastic measurement error we consider the estimation of the regression of an outcome y on a covariate x where x is unobserved but a variable w that measures x with error is observed a calibration sample that measures pairs of values of x and w is also available we consider calibration samples where y is measured internal calibration and not measured external calibration one common approach for measurement error correction is regression calibration rc which substitutes the unknown values of x by predictions from the regression of x on w estimated from the calibration sample an alternative approach is to multiply impute the missing values of x given y and w based on an imputation model and then use multiple imputation mi combining rules for inferences most of current work assumes that the measurement error of w has a constant variance whereas in many situations the variance varies as a function of x we consider extensions of the rc and mi methods that allow for heteroscedastic measurement error and compare them by simulation the mi method is shown to provide better inferences in this setting we also illustrate the proposed methods using a data set from the biocycle study

uniform confidence bands for nonparametric errors in variables regression abstract this paper develops a method to construct uniform confidence bands for a nonparametric regression function where a predictor variable is subject to a measurement error we allow for the distribution of the measurement error to be unknown but assume the availability of validation data or repeated measurements on the latent predictor variable the proposed confidence band builds on the deconvolution kernel estimation and a novel application of the multiplier bootstrap method we establish asymptotic validity of the proposed confidence band to our knowledge this is the first paper to derive asymptotically valid uniform confidence bands for nonparametric errors in variables regression

on the optimal rates of convergence for nonparametric deconvolution problems deconvolution problems arise in a variety of situations in statistics an interesting problem is to estimate the density f of a random variable x based on n i i d observations from y x varepsilon where varepsilon is a measurement error with a known distribution in this paper the effect of errors in variables of nonparametric deconvolution is examined insights are gained by showing that the difficulty of deconvolution depends on the smoothness of error distributions the smoother the harder in fact there are two types of optimal rates of convergence according to whether the error distribution is ordinary smooth or supersmooth it is shown that optimal rates of convergence can be achieved by deconvolution kernel density estimators

nonparametric regression with errors in variables the effect of errors in variables in nonparametric regression estimation is examined to account for errors in covariates deconvolution is involved in the construction of a new class of kernel estimators it is shown that optimal local and global rates of convergence of these kernel estimators can be characterized by the tail behavior of the characteristic function of the error distribution in fact there are two types of rates of convergence according to whether the error is ordinary smooth or super smooth it is also shown that these results hold uniformly over a class of joint distributions of the response and the covariate which is rich enough for many practical applications furthermore to achieve optimality we show that the convergence rates of all possible estimators have a lower bound possessed by the kernel estimators

semi supervised local multi manifold isomap by linear embedding for feature extraction we explore the discriminative feature extraction problem a semisupervised local multimanifold isomap by linear embedding is proposed our model can use labeled and unlabeled data to deliver manifold features our model aims to minimize pairwise intraclass distances in the same manifold our model aims to maximize the distances between different manifolds in this paper we mainly propose a semi supervised local multi manifold isomap learning framework by linear embedding termed ssmm isomap that can apply the labeled and unlabeled training samples to perform the joint learning of neighborhood preserving local nonlinear manifold features and a linear feature extractor the formulation of ssmm isomap aims at minimizing pairwise distances of intra class points in the same manifold and maximizing the distances over different manifolds to enhance the performance of nonlinear manifold feature learning we also incorporate the neighborhood reconstruction error to preserve local topology structures between both labeled and unlabeled samples to enable our ssmm isomap to extract local manifold features from the outside new data we also add a feature approximation error that correlates manifold features with embedded features by the jointly learnt feature extractor thus the learnt linear extractor can extract the local manifold features from the new data efficiently by direct embedding to optimize the proposed objective function two effective schemes are presented i e scaling by majorizing a complicated function and eigen decomposition notice that the comparison of the proposed two solvers is also described we mainly evaluate ssmm isomap for manifold feature learning data clustering and classification extensive simulation results verify the effectiveness of our ssmm isomap algorithm compared with other related feature learning techniques

covariate measurement error in logistic regression abstract in a logistic regression model when covariates are subject to measurement error the naive estimator obtained by regressing on the observed covariates is asymptotically biased this document introduces a bias adjusted estimator and two estimators appropriate for normally distributed measurement errors a functional maximum likelihood estimator and an estimator which exploits the consequences of sufficiency the four proposals are studied asymptotically under conditions which are appropriate when the measurement error is small a small monte carlo study illustrates the superiority of the measurement error estimators in certain situations additional keywords mathematical models

low rank preserving embedding abstract in this paper we consider the problem of linear dimensionality reduction with the novel technique of low rank representation which is a promising tool of discovering subspace structures of given data existing approaches based on graph embedding usually capture structure of data via stacking the local structure of each datum such as neighborhood graph l1 graph and l2 graph yet they lack explicit discrimination between those local structures and suffer from corrupted samples to this end we propose a new linear dimensionality reduction method by virtue of the lowest rank representation lrr of data which is dubbed low rank preserving embedding lrpe different from the traditional routes lrpe achieves all data self representations jointly and can thus extract the global structure of a data set as a whole the global low rank constraint explicitly enforces the lrr matrix to be block diagonal form so that the samples with a similar intrinsic structure which are more likely to be from the same class are described by a similar set of bases hence lrpe is discriminative even if no class labels are provided benefiting from the robust lrr lrpe is also robust to various noises and errors in data besides we rewritten all related methods into a unified formulation followed by a detailed solution and clear comparisons finally we conduct extensive experiments on publicly available data sets for data visualization and classification the inspiring experimental results show the effectiveness the cheap computation and the robustness of the proposed method

recursive identification of errors in variables wiener systems this paper considers the recursive identification of errors in variables eiv wiener systems composed of a linear dynamic system followed by a static nonlinearity both the system input and output are observed with additive noises being arma processes with unknown coefficients by a stochastic approximation incorporated with the deconvolution kernel functions the recursive algorithms are proposed for estimating the coefficients of the linear subsystem and for the values of the nonlinear function all the estimates are proved to converge to the true values with probability one a simulation example is given to verify the theoretical analysis

confidence bands in non parametric errors in variables regression summary errors in variables regression is important in many areas of science and social science e g in economics where it is often a feature of hedonic models in environmental science where air quality indices are measured with error in biology where the vegetative mass of plants is frequently obscured by mismeasurement and in nutrition where reported fat intake is typically subject to substantial error to date in non parametric contexts the great majority of work has focused on methods for estimating the mean as a function with relatively little attention being paid to techniques for empirical assessment of the accuracy of the estimator we develop methodologies for constructing confidence bands our contributions include techniques for tuning parameter choice aimed at minimizing the coverage error of confidence bands

the implicit discretization of the supertwisting sliding mode control algorithm this article deals with the analysis of the time discretization of the supertwisting algorithm with an implicit euler method it is shown that the discretized system is well posed the existence of a lyapunov function with convex level sets is proved for the continuous time closed loop system then the global asymptotic lyapunov stability of the unperturbed discrete time closed loop system is proved the convergence to the origin in a finite number of steps is proved also in the unperturbed case numerical simulations demonstrate the superiority of the implicit method with respect to an explicit discretization with significant chattering reduction