regression on manifolds estimation of the exterior derivative collinearity and near collinearity of predictors cause difficulties when doing regression in these cases variable selection becomes untenable because of mathematical issues concerning the existence and numerical stability of the regression coefficients and interpretation of the coefficients is ambiguous because gradients are not defined using a differential geometric interpretation in which the regression coefficients are interpreted as estimates of the exterior derivative of a function we develop a new method to do regression in the presence of collinearities our regularization scheme can improve estimation error and it can be easily modified to include lasso type regularization these estimators also have simple extensions to the large p small n context

covariance regularization by thresholding this paper considers regularizing a covariance matrix of p variables estimated from n observations by hard thresholding we show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense the variables are gaussian or sub gaussian and log p n 0 and obtain explicit rates the results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity we discuss an intuitive resampling scheme for threshold selection and prove a general cross validation result that justifies this approach we also compare thresholding to other covariance estimators in simulations and on an example from climate data 1 introduction estimation of covariance matrices is important in a number of areas of statistical analysis including dimension reduction by principal component analysis pca classification by linear or quadratic discriminant analysis lda and qda establishing independence and conditional independence relations in the context of graphical models and setting confidence intervals on linear functions of the means of the components in recent years many application areas where these tools are used have been dealing with very high dimensional datasets and sample sizes can be very small relative to dimension examples include genetic data brain imaging spectroscopic imaging climate data and many others it is well known by now that the empirical covariance matrix for samples of size n from a p variate gaussian distribution np p is not a good estimator of the population covariance if p is large many results in random matrix theory illustrate this from the classical mary law 29 to the more recent work of johnstone and his students on the theory of the largest eigenvalues 12 23 30 and associated eigenvectors 24 however with the exception of a method for estimating the covariance spectrum 11 these probabilistic results do not offer alternatives to the sample covariance matrix alternative estimators for large covariance matrices have therefore attracted a lot of attention recently two broad classes of covariance estimators have emerged those that rely on a natural ordering among variables and assume that variables

existence of efficient and properly efficient solutions to problems of constrained vector optimization the paper is devoted to the existence of global optimal solutions for a general class of nonsmooth problems of constrained vector optimization without boundedness assumptions on constraint set the main attention is paid to the two major notions of optimality in vector problems pareto efficiency and proper efficiency in the sense of geoffrion employing adequate tools of variational analysis and generalized differentiation we first establish relationships between the notions of properness m tameness and the palais smale conditions formulated for the restriction of the vector cost mapping on the constraint set these results are instrumental to derive verifiable necessary and sufficient conditions for the existence of pareto efficient solutions in vector optimization furthermore the developed approach allows us to obtain new sufficient conditions for the existence of geoffrion properly efficient solutions to such constrained vector problems

on consistency and sparsity for principal components analysis in high dimensions principal components analysis pca is a classic method for the reduction of dimensionality of data in the form of n observations or cases of a vector with p variables contemporary datasets often have p comparable with or even much larger than n our main assertions in such settings are a that some initial reduction in dimensionality is desirable before applying any pca type search for principal modes and b the initial reduction in dimensionality is best achieved by working in a basis in which the signals have a sparse representation we describe a simple asymptotic model in which the estimate of the leading principal component vector via standard pca is consistent if and only if p n n 0 we provide a simple algorithm for selecting a subset of coordinates with largest sample variances and show that if pca is done on the selected subset then consistency is recovered even if p n n

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

a pre order principle and set valued ekeland variational principle abstract we establish a pre order principle from the principle we obtain a very general set valued ekeland variational principle where the objective function is a set valued map taking values in a quasi ordered linear space and the perturbation contains a family of set valued maps satisfying certain property from this general set valued ekeland variational principle we deduce a number of particular versions of set valued ekeland variational principle which include many known ekeland variational principles their improvements and some new results

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

a new penalization tool in scalar and vector optimizations abstract in this work we employ a new method to penalize scalar and vector optimization problems by means of a functional known in the literature as the directional minimal time function we show that this new approach is useful also for the strong pareto optimality and then we prove for this kind of solutions a result concerning necessary optimality conditions expressed in terms of mordukhovich generalized differentiation

multivariate locally weighted least squares regression nonparametric regression using locally weighted least squares was first discussed by stone and by cleveland recently it was shown by fan and by fan and gijbels that the local linear kernel weighted least squares regression estimator has asymptotic properties making it superior in certain senses to the nadaraya watson and gasser muller kernel estimators in this paper we extend their results on asymptotic bias and variance to the case of multivariate predictor variables we are able to derive the leading bias and variance terms for general multivariate kernel weights using weighted least squares matrix theory this approach is especially convenient when analysing the asymptotic conditional bias and variance of the estimator at points near the boundary of the support of the predictors we also investigate the asymptotic properties of the multivariate local quadratic least squares regression estimator discussed by cleveland and devlin and in the univariate case higher order polynomial fits and derivative estimation

the lagrange multiplier rule for multifunctions in banach spaces we study general constrained multiobjective optimization problems with objectives being closed multifunctions in banach spaces in terms of the coderivatives and normal cones we provide generalized lagrange multiplier rules as necessary optimality conditions of the above problems in an asplund space setting sharper results are presented

the fermat rule for multifunctions on banach spaces using variational analysis we study vector optimization problems with objectives being closed multifunctions on banach spaces or in asplund spaces in particular in terms of the coderivatives we present fermat s rules as necessary conditions for an optimal solution of the above problems as applications we also provide some necessary conditions in terms of clarke s normal cones or the limiting normal cones for pareto efficient points

tangential extremal principles for finite and infinite systems of sets ii applications to semi infinite and multiobjective optimization this paper contains selected applications of the new tangential extremal principles and related results developed in mordukhovich and phan math program 2011 to calculus rules for infinite intersections of sets and optimality conditions for problems of semi infinite programming and multiobjective optimization with countable constraints

the directional instability problem in systems with frictional contacts the research summarized in this paper addresses the directional instability of finite dimensional systems with unilateral frictional contacts conditions for the occurrence of this divergence type instability are discussed complementarity formulations are developed and numerical procedures are proposed for the solution of the corresponding non smooth stability eigenproblems various examples are analytically or numerically solved and discussed namely some finite element examples that have instability modes involving evolution towards slip or stick in different portions of the contact surface

semidefinite programming relaxations for semialgebraic problems a hierarchy of convex relaxations for semialgebraic problems is introduced for questions reducible to a finite number of polynomial equalities and inequalities it is shown how to construct a complete family of polynomially sized semidefinite programming conditions that prove infeasibility the main tools employed are a semidefinite programming formulation of the sum of squares decomposition for multivariate polynomials and some results from real algebraic geometry the techniques provide a constructive approach for finding bounded degree solutions to the positivstellensatz and are illustrated with examples from diverse application fields

partially finite convex programming part ii explicit lattice models in part i of this work we derived a duality theorem for partially finite convex programs problems for which the standard slater condition fails almost invariably our result depended on a constraint qualification involving the notion ofquasi relative interior the derivation of the primal solution from a dual solution depended on the differentiability of the dual objective function the differentiability of various convex functions in lattices was considered at the end of part i in part ii we shall apply our results to a number of more concrete problems including variants of semi infinite linear programming l1 approximation constrained approximation and interpolation spectral estimation semi infinite transportation problems and the generalized market area problem of lowe and hurter 1976 as in part i we shall use lattice notation extensively but as we illustrated there in concrete examples lattice theoretic ideas can be avoided if preferred by direct calculation

on the concept of generalized order optimality abstract the main results of this paper include a detailed analysis of the notion of generalized order optimality and some sufficient conditions for a point satisfying the necessary optimality condition of mordukhovich 2006 1 2 for being a generalized order solution of the optimization problem under consideration

nonparametric regression in the presence of measurement error summary in many regression applications the independent variable is measured with error when this happens conventional parametric and nonparametric regression techniques are no longer valid we consider two diverent approaches to nonparametric regression the first uses the simex simulation extrapolation method and makes no assumption about the distribution of the unobserved error prone predictor for this approach we derive an asymptotic theory for kernel regression which has some surprising implications penalised regression splines are also considered for fixed number of known knots the second approach assumes that the error prone predictor has a distribution of a mixture of normals with an unknown number of components and uses regression splines simulations illustrate the results

constrained estimators and consistency of a regression model on a lexis diagram this article considers a regression model on a lexis diagram of an a p table with a single response in each cell following a distribution in the exponential family a regression model on the fixed effects of a rows p columns and a p 1 diagonals induces a singular design matrix and yields multiple estimators leading to parameter identifiability problem in age period cohort analysis in social sciences demography and epidemiology where assessment of secular trend in age period and birth cohort of social events e g violence and diseases e g cancer is of interest similar problems also exist in other settings such as in supersaturated designs in this article we study the finite sample properties of the multiple estimators propose a penalized profile likelihood method to study the consistency and asymptotic bias and demonstrate the results through simulations and data analysis as a by product the identifiability problem is addressed with consistent estimation for model parameters an

some asymptotic theory for the bootstrap efron s bootstrap method of distribution approximation is shown to be asymptotically valid in a large number of situations including t statistics the empirical and quantile processes and von mises functionals some counter examples are also given to show that the approximation does not always succeed

linear model selection by cross validation abstract we consider the problem of selecting a model having the best predictive ability among a class of linear models the popular leave one out cross validation method which is asymptotically equivalent to many other model selection methods such as the akaike information criterion aic the c p and the bootstrap is asymptotically inconsistent in the sense that the probability of selecting the model with the best predictive ability does not converge to 1 as the total number of observations n we show that the inconsistency of the leave one out cross validation can be rectified by using a leave n v out cross validation with n v the number of observations reserved for validation satisfying n v n 1 as n this is a somewhat shocking discovery because nv n 1 is totally opposite to the popular leave one out recipe in cross validation motivations justifications and discussions of some practical aspects of the use of the leave n v out cross validation method are provided and results

copositivity and the minimization of quadratic functions with nonnegativity and quadratic equality constraints the problem of finding the minimum value of a quadratic function on a set defined by nonnegativity and quadratic equality constraints is analyzed the difficulty in finding the solution to this problem is primarily due to the fact that the feasible region is nonconvex an algorithm that requires the hessian of the quadratic constraint function be strictly copositive is developed for finding the minimal value of the quadratic objective function the problem of finding this global minima can be mapped into the problem of determining whether or not a particular matrix is copositive this result is equivalent to earlier results characterizing the solutions to a large class of fractional programming problems a more efficient algorithm for finding solutions that satisfy the kuhn tucker necessary conditions is developed and its convergence behavior is analyzed this algorithm requires that the hessians of the quadratic constraint and objective functions be both positive semidefinite and strictly copositive

e strict subdifferentials of set valued maps and optimality conditions abstract in this paper first a new notion of e strict subdifferentials of set valued maps is introduced in a locally convex space second the existence for a e strict subdifferential of a set valued map and an equivalent characterization for e strict subgradient of a set valued map are presented respectively third a generalized moreau rockafellar theorem with set valued maps is obtained finally optimality conditions of vector optimization problems with set valued maps are established in the sense of e strict subdifferential

on the existence of pareto solutions for polynomial vector optimization problems we are interested in the existence of pareto solutions to the vector optimization problem begin aligned mathrm min mathbb r m f x x in mathbb r n end aligned where f mathbb r n rightarrow mathbb r m is a polynomial map by using the tangency variety of f we first construct a semi algebraic set of dimension at most m 1 containing the set of pareto values of the problem then we establish connections between the palais smale conditions m tameness and properness for the map f based on these results we provide some sufficient conditions for the existence of pareto solutions of the problem we also introduce a generic class of polynomial vector optimization problems having at least one pareto solution

operator norm consistent estimation of large dimensional sparse covariance matrices estimating covariance matrices is a problem of fundamental importance in multivariate statistics in practice it is increasingly frequent to work with data matrices x of dimension n x p where p and n are both large results from random matrix theory show very clearly that in this setting standard estimators like the sample covariance matrix perform in general very poorly in this large n large p setting it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0 and hence this matrix is sparse we develop an estimator to handle this situation the estimator is shown to be consistent in operator norm when for instance we have p n as n oo in other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero this implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues we also propose a notion of sparsity for matrices that is compatible with spectral analysis and is independent of the ordering of the variables

a branch and cut algorithm for nonconvex quadratically constrained quadratic programming we present a branch and cut algorithm that yields in finite time a globally e optimal solution with respect to feasibility and optimality of the nonconvex quadratically constrained quadratic programming problem the idea is to estimate all quadratic terms by successive linearizations within a branching tree using reformulation linearization techniques rlt to do so four classes of linearizations cuts depending on one to three parameters are detailed for each class we show how to select the best member with respect to a precise criterion the cuts introduced at any node of the tree are valid in the whole tree and not only within the subtree rooted at that node in order to enhance the computational speed the structure created at any node of the tree is flexible enough to be used at other nodes computational results are reported that include standard test problems taken from the literature some of these problems are solved for the first time with a proof of global optimality

the vector valued variational principle in banach spaces ordered by cones with nonempty interiors we prove sharpness of efficient solutions x k to vector optimization problems resulting from ekeland vector variational principles we achieve this by sharpening some of the existing vector variational principles and showing that x k remains efficient not only for perturbations in the direction k but also for other directions of perturbations

relative pareto minimizers for multiobjective problems existence and optimality conditions in this paper we introduce and study enhanced notions of relative pareto minimizers for constrained multiobjective problems that are defined via several kinds of relative interiors of ordering cones and occupy intermediate positions between the classical notions of pareto and weak pareto efficiency minimality using advanced tools of variational analysis and generalized differentiation we establish the existence of relative pareto minimizers for general multiobjective problems under a refined version of the subdifferential palais smale condition for set valued mappings with values in partially ordered spaces and then derive necessary optimality conditions for these minimizers as well as for conventional efficient and weak efficient counterparts that are new in both finite dimensional and infinite dimensional settings our proofs are based on variational and extremal principles of variational analysis in particular on new versions of the ekeland variational principle and the subdifferential variational principle for set valued and single valued mappings in infinite dimensional spaces

the ekeland variational principle for henig proper minimizers and super minimizers abstract in this paper we consider for the first time approximate henig proper minimizers and approximate super minimizers of a set valued map f with values in a partially ordered vector space and formulate two versions of the ekeland variational principle for these points involving coderivatives in the sense of ioffe clarke and mordukhovich as applications we obtain sufficient conditions for f to have a henig proper minimizer or a super minimizer under the palais smale type conditions the techniques are essentially based on the characterizations of henig proper efficient points and super efficient points by mean of the henig dilating cones and the hiriart urruty signed distance function

copositivity and constrained fractional quadratic problems we provide completely positive and copositive optimization formulations for the constrained fractional quadratic problem cfqp and standard fractional quadratic problem stfqp based on these formulations semidefinite programming relaxations are derived for finding good lower bounds to these fractional programs which can be used in a global optimization branch and bound approach applications of the cfqp and stfqp related with the correction of infeasible linear systems and eigenvalue complementarity problems are also discussed

on the asymptotics of constrained m estimation limit theorems for an m estimate constrained to lie in a closed subset of r d are given under two different sets of regularity conditions a consistent sequence of global optimizers converges under chernoff regularity of the parameter set a n consistent sequence of local optimizers converges under clarke regularity of the parameter set in either case the asymptotic distribution is a projection of a normal random vector on the tangent cone of the parameter set at the true parameter value limit theorems for the optimal value are also obtained agreeing with chernoff s result in the case of maximum likelihood with global optimizers

local linear regression on manifolds and its geometric interpretation high dimensional data analysis has been an active area and the main focus areas have been variable selection and dimension reduction in practice it occurs often that the variables are located on an unknown lower dimensional nonlinear manifold under this manifold assumption one purpose of this article is regression and gradient estimation on the manifold and another is developing a new tool for manifold learning as regards the first aim we suggest directly reducing the dimensionality to the intrinsic dimension d of the manifold and performing the popular local linear regression llr on a tangent plane estimate an immediate consequence is a dramatic reduction in the computational time when the ambient space dimension p d we provide rigorous theoretical justification of the convergence of the proposed regression and gradient estimators by carefully analyzing the curvature boundary and nonuniform sampling effects we propose a bandwidth selector that can handle heteroscedastic errors with re

algorithmic copositivity detection by simplicial partition we present new criteria for copositivity of a matrix i e conditions which ensure that the quadratic form induced by the matrix is nonnegative over the nonnegative orthant these criteria arise from the representation of the quadratic form in barycentric coordinates with respect to the standard simplex and simplicial partitions thereof we show that as the partition gets finer and finer the conditions eventually capture all strictly copositive matrices we propose an algorithmic implementation which considers several numerical aspects as an application we present results on the maximum clique problem we also briefly discuss extensions of our approach to copositivity with respect to arbitrary polyhedral cones

overfitting in making comparisons between variable selection methods this paper addresses a common methodological flaw in the comparison of variable selection methods a practical approach to guide the search or the selection process is to compute cross validation performance estimates of the different variable subsets used with computationally intensive search algorithms these estimates may overfit and yield biased predictions therefore they cannot be used reliably to compare two selection methods as is shown by the empirical results of this paper instead like in other instances of the model selection problem independent test sets should be used for determining the final performance the claims made in the literature about the superiority of more exhaustive search algorithms over simpler ones are also revisited and some of them infirmed

the topology of probability distributions on manifolds let mathcal p be a set of n random points in mathbb r d generated from a probability measure on a m dimensional manifold mathcal m subset mathbb r d in this paper we study the homology of mathcal u mathcal p r the union of d dimensional balls of radius r around mathcal p as n rightarrow infty and r rightarrow 0 in addition we study the critical points of d mathcal p the distance function from the set mathcal p these two objects are known to be related via morse theory we present limit theorems for the betti numbers of mathcal u mathcal p r as well as for number of critical points of index k for d mathcal p depending on how fast r decays to zero as n grows these two objects exhibit different types of limiting behavior in one particular case n r m ge c log n we show that the betti numbers of mathcal u mathcal p r perfectly recover the betti numbers of the original manifold mathcal m a result which is of significant interest in topological manifold learning

a optimal projection for image representation we consider the problem of image representation from the perspective of statistical design recent studies have shown that images are possibly sampled from a low dimensional manifold despite of the fact that the ambient space is usually very high dimensional learning low dimensional image representations is crucial for many image processing tasks such as recognition and retrieval most of the existing approaches for learning low dimensional representations such as principal component analysis pca and locality preserving projections lpp aim at discovering the geometrical or discriminant structures in the data in this paper we take a different perspective from statistical experimental design and propose a novel dimensionality reduction algorithm called a optimal projection aop aop is based on a linear regression model specifically aop finds the optimal basis functions so that the expected prediction error of the regression model can be minimized if the new representations are used for training the model experimental results suggest that the proposed approach provides a better representation and achieves higher accuracy in image retrieval

generalized e quasi solutions in multiobjective optimization problems existence results and optimality conditions abstract this paper is concerned with approximate solutions of multiobjective optimization problems whose order cone is not necessarily the nonnegative orthant we introduce the concept of generalized e quasi solution that extends other well known approximate efficiency notions of the literature and we study the limit behavior of these solutions as approximations to the efficient and weak efficient sets moreover we prove several existence results and a bound for the generalized e quasi solution set under convexity assumptions and by using asymptotic analysis tools finally we develop optimality conditions for a particular case of these kinds of e quasi solutions in nonsmooth convex and nonconvex problems

multiobjective fractional variational calculus in terms of a combined caputo derivative abstract the study of fractional variational problems in terms of a combined fractional caputo derivative is introduced necessary optimality conditions of euler lagrange type for the basic isoperimetric and lagrange variational problems are proved as well as transversality and sufficient optimality conditions this allows to obtain necessary and sufficient pareto optimality conditions for multiobjective fractional variational problems

manifold learning the price of normalization we analyze the performance of a class of manifold learning algorithms that find their output by minimizing a quadratic form under some normalization constraints this class consists of locally linear embedding lle laplacian eigenmap local tangent space alignment ltsa hessian eigenmaps hlle and diffusion maps we present and prove conditions on the manifold that are necessary for the success of the algorithms both the finite sample case and the limit case are analyzed we show that there are simple manifolds in which the necessary conditions are violated and hence the algorithms cannot recover the underlying manifolds finally we present numerical results that demonstrate our claims

new and old bounds for standard quadratic optimization dominance equivalence and incomparability a standard quadratic optimization problem stqp consists in minimizing a quadratic form over a simplex among the problems which can be transformed into a stqp are the general quadratic problem over a polytope and the maximum clique problem in a graph in this paper we present several new polynomial time bounds for stqp ranging from very simple and cheap ones to more complex and tight constructions the main tools employed in the conception and analysis of most bounds are semidefinite programming and decomposition of the objective function into a sum of two quadratic functions each of which is easy to minimize we provide a complete diagram of the dominance incomparability or equivalence relations among the bounds proposed in this and in previous works in particular we show that one of our new bounds dominates all the others furthermore a specialization of such bound dominates schrijver s improvement of lovasz s function bound for the maximum size of a clique in a graph

statistics with set valued functions applications to inverse approximate optimization much of statistics relies upon four key elements a law of large numbers a calculus to operationalize stochastic convergence a central limit theorem and a framework for constructing local approximations these elements are well understood for objects in a vector space e g points or functions however much statistical theory does not directly translate to sets because they do not form a vector space building on probability theory for random sets this paper uses variational analysis to develop operational tools for statistics with set valued functions these tools are first applied to nonparametric estimation kernel regression of set valued functions the second application is to the problem of inverse approximate optimization in which approximate solutions corrupted by noise to an optimization problem are observed and then used to estimate the amount of suboptimality of the solutions and the parameters of the optimization problem that generated the solutions we show that previous approaches to this problem are statistically inconsistent when the data is corrupted by noise whereas our approach is consistent under mild conditions

geodesic entropic graphs for dimension and entropy estimation in manifold learning in the manifold learning problem one seeks to discover a smooth low dimensional surface i e a manifold embedded in a higher dimensional linear vector space based on a set of measured sample points on the surface in this paper we consider the closely related problem of estimating the manifold s intrinsic dimension and the intrinsic entropy of the sample points specifically we view the sample points as realizations of an unknown multivariate density supported on an unknown smooth manifold we introduce a novel geometric approach based on entropic graph methods although the theory presented applies to this general class of graphs we focus on the geodesic minimal spanning tree gmst to obtaining asymptotically consistent estimates of the manifold dimension and the re spl acute nyi spl alpha entropy of the sample density on the manifold the gmst approach is striking in its simplicity and does not require reconstruction of the manifold or estimation of the multivariate density of the samples the gmst method simply constructs a minimal spanning tree mst sequence using a geodesic edge matrix and uses the overall lengths of the msts to simultaneously estimate manifold dimension and entropy we illustrate the gmst approach on standard synthetic manifolds as well as on real data sets consisting of images of faces

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

nonlinear versions of a vector maximal principle abstract some nonlinear extensions of the vector maximality statement established by goepfert et al a goepfert c tammer c z linescu on the vectorial ekeland s variational principle and minimal points in product spaces nonlinear anal 39 2000 909 922 are given basic instruments for these are the brezis browder ordering principle h brezis f e browder a general principle on ordered sets in nonlinear functional analysis adv math 21 1976 355 364 and its logical equivalent in turinici m turinici variational principles on semi metric structures libertas math 20 2000 161 171

joint harmonic functions and their supervised connections the cluster assumption had a significant impact on the reasoning behind semi supervised classification methods in graph based learning the literature includes numerous applications where harmonic functions provided estimates that conformed to data satisfying this well known assumption but the relationship between this assumption and harmonic functions is not as well understood theoretically we investigate these matters from the perspective of supervised kernel classification and provide concrete answers to two fundamental questions i under what conditions do semi supervised harmonic approaches satisfy this assumption ii if such an assumption is satisfied then why precisely would an observation sacrifice its own supervised estimate in favor of the cluster first a harmonic function is guaranteed to assign labels to data in harmony with the cluster assumption if a specific condition on the boundary of the harmonic function is satisfied second it is shown that any harmonic function estimate within the interior is a probability weighted average of supervised estimates where the weight is focused on supervised kernel estimates near labeled cases we demonstrate that the uniqueness criterion for harmonic estimators is sensitive when the graph is sparse or the size of the boundary is relatively small this sets the stage for a third contribution a new regularized joint harmonic function for semi supervised learning based on a joint optimization criterion mathematical properties of this estimator such as its uniqueness even when the graph is sparse or the size of the boundary is relatively small are proven a main selling point is its ability to operate in circumstances where the cluster assumption may not be fully satisfied on real data by compromising between the purely harmonic and purely supervised estimators the competitive stature of the new regularized joint harmonic approach is established

variational formulation for a general dynamic financial equilibrium problem balance law and liability formula abstract an evolutionary model for a multi sector multi instrument financial equilibrium problem with a general utility function different prices for assets and liabilities and including the expenses for the management of the financial institutions is presented in this general case we give the evolutionary financial equilibrium conditions and prove their equivalence with an evolutionary variational inequality from which existence results of the financial equilibrium follow moreover making use of new nonlinear analysis results we develop a lagrange theory which allows one to study the behaviour of the financial equilibrium by means of the lagrange multipliers as a product of this analysis we prove that for the financial model an equilibrium law together with a liability formula must be fulfilled from which the reorganization of the existing financial disequilibrium depends

scalarization and optimality conditions for vector equilibrium problems abstract in this paper we investigated vector equilibrium problems and gave the scalarization results for weakly efficient solutions henig efficient solutions and globally efficient solutions to the vector equilibrium problems without the convexity assumption using nonsmooth analysis and the scalarization results we provided the necessary conditions for weakly efficient solutions henig efficient solutions globally efficient solutions and superefficient solutions to vector equilibrium problems by the assumption of convexity we gave sufficient conditions for those solutions as applications we gave the necessary and sufficient conditions for corresponding solutions to vector variational inequalities and vector optimization problems

optimality and duality for proper and isolated efficiencies in multiobjective optimization we use some advanced tools of variational analysis and generalized differentiation such as the nonsmooth version of fermat s rule the limiting mordukhovich subdifferential of maximum functions and the sum rules for the frechet subdifferential and for the limiting one to establish necessary conditions for local properly efficient solutions and local isolated minimizers of a multiobjective optimization problem involving inequality and equality constraints sufficient conditions for the existence of such solutions are also provided under assumptions of local convex affine functions or l invex infine functions defined in terms of the limiting subdifferential of locally lipschitz functions in addition we propose a type of wolfe dual problems and examine weak strong duality relations under l invexity infineness hypotheses

nonparametric regression for spherical data we develop nonparametric smoothing for regression when both the predictor and the response variables are defined on a sphere of whatever dimension a local polynomial fitting approach is pursued which retains all the advantages in terms of rate optimality interpretability and ease of implementation widely observed in the standard setting our estimates have a multi output nature meaning that each coordinate is separately estimated within a scheme of a regression with a linear response the main properties include linearity and rotational equivariance this research has been motivated by the fact that very few models describe this kind of regression such current methods are surely not widely employable since they have a parametric nature and also require the same dimensionality for prediction and response spaces along with nonrandom design our approach does not suffer these limitations real data case studies and simulation experiments are used to illustrate the effectiveness of the method

bootstrap model selection abstract in a regression problem typically there are p explanatory variables possibly related to a response variable and we wish to select a subset of the p explanatory variables to fit a model between these variables and the response a bootstrap variable model selection procedure is to select the subset of variables by minimizing bootstrap estimates of the prediction error where the bootstrap estimates are constructed based on a data set of size n although the bootstrap estimates have good properties this bootstrap selection procedure is inconsistent in the sense that the probability of selecting the optimal subset of variables does not converge to 1 as n this inconsistency can be rectified by modifying the sampling method used in drawing bootstrap observations for bootstrapping pairs response explanatory variable it is found that instead of drawing n bootstrap observations a customary bootstrap sampling plan much less bootstrap observations should be sampled the bootstrap selection p

necessary suboptimality and optimality conditions via variational principles the paper aims to develop some basic principles and tools of nonconvex variational analysis with applications to necessary suboptimality and optimality conditions for constrained optimization problems in infinite dimensions we establish a certain subdifferential variational principle as a new characterization of asplund spaces this result is different from conventional support forms of variational principles and appears to be convenient for applications to nonsmooth optimization based on the subdifferential variational principle we obtain new necessary conditions for suboptimal solutions in general nonsmooth optimization problems with equality inequality and set constraints in asplund spaces in this way we establish the so called sequential normal compactness properties of constraint sets that play an essential role in infinite dimensional variational analysis and its applications as a by product of our approach we derive various forms of necessary optimality conditions for nonsmooth constrained problems in infinite dimensions which extend known results in that direction

global optimization with polynomials and the problem of moments we consider the problem of finding the unconstrained global minimum of a real valued polynomial p x mathbb r n to mathbb r as well as the global minimum of p x in a compact set k defined by polynomial inequalities it is shown that this problem reduces to solving an often finite sequence of convex linear matrix inequality lmi problems a notion of karush kuhn tucker polynomials is introduced in a global optimality condition some illustrative examples are provided

strict approximate solutions in set valued optimization with applications to the approximate ekeland variational principle this work deals with strict solutions of set valued optimization problems under the set optimality criterion in this context we introduce a new approximate solution concept and we obtain several properties of these solutions when the error is fixed and also for their limit behavior when the error tends to zero then we prove a general existence result which is applied to obtain approximate ekeland variational principles

exact penalty property in optimization with mixed constraints via variational analysis in this paper we use the penalty approach in order to study constrained minimization problems in infinite dimensional asplund spaces with nonsmooth nonconvex mixed constraints a penalty function is said to have the exact penalty property if there is a penalty coefficient for which a solution of an unconstrained penalized problem is a solution of the corresponding constrained problem in this paper we establish sufficient conditions for the exact penalty properties using the notion of the mordukhovich basic subdifferential

ekeland s variational principle for set valued functions we establish several set valued function versions of ekeland s variational principle and hence provide some sufficient conditions ensuring the existence of error bounds for inequality systems defined by finitely many lower semicontinuous functions

a unified separation theorem for closed sets in a banach space and optimality conditions for vector optimization using the technique of variational analysis and in terms of normal cones we establish unified separation results for finitely many closed not necessarily convex sets in banach spaces which not only cover the existing nonconvex separation results and a classical convex separation theorem but also recapture the approximate projection theorem with help of the separation result for closed sets we provide necessary and sufficient conditions for approximate pareto solutions of constrained vector optimization problems in particular we extend some basic optimality results for approximate solutions of numerical optimization problems to the vector optimization setting

on semi supervised linear regression in covariate shift problems semi supervised learning approaches are trained using the full training labeled data and available testing unlabeled data demonstrations of the value of training with unlabeled data typically depend on a smoothness assumption relating the conditional expectation to high density regions of the marginal distribution and an inherent missing completely at random assumption for the labeling so called covariate shift poses a challenge for many existing semi supervised or supervised learning techniques covariate shift models allow the marginal distributions of the labeled and unlabeled feature data to differ but the conditional distribution of the response given the feature data is the same an example of this occurs when a complete labeled data sample and then an unlabeled sample are obtained sequentially as it would likely follow that the distributions of the feature data are quite different between samples the value of using unlabeled data during training for the elastic net is justified geometrically in such practical covariate shift problems the approach works by obtaining adjusted coefficients for unlabeled prediction which recalibrate the supervised elastic net to compromise i maintaining elastic net predictions on the labeled data with ii shrinking unlabeled predictions to zero our approach is shown to dominate linear supervised alternatives on unlabeled response predictions when the unlabeled feature data are concentrated on a low dimensional manifold away from the labeled data and the true coefficient vector emphasizes directions away from this manifold large variance of the supervised predictions on the unlabeled set is reduced more than the increase in squared bias when the unlabeled responses are expected to be small so an improved compromise within the bias variance tradeoff is the rationale for this performance improvement performance is validated on simulated and real data

a new necessary and sufficient condition for the strong duality and the infinite dimensional lagrange multiplier rule abstract throughout this paper the authors introduce a new condition defined by assumption s which establishes a necessary and sufficient condition for the validity of the strong duality between a convex optimization problem and its lagrange dual this work will be focused on the context of emptiness of the interior of the ordering cone and convexity of the equality constraints moreover this new condition will be necessary and sufficient for the infinite dimensional lagrange multiplier rule this new principle will find application to the elastic plastic torsion problem to the continuum model of transportation and to a problem with quadratic equality constraint with connected to evolutionary illumination and visibility problems

nonparametric regression with errors in variables the effect of errors in variables in nonparametric regression estimation is examined to account for errors in covariates deconvolution is involved in the construction of a new class of kernel estimators it is shown that optimal local and global rates of convergence of these kernel estimators can be characterized by the tail behavior of the characteristic function of the error distribution in fact there are two types of rates of convergence according to whether the error is ordinary smooth or super smooth it is also shown that these results hold uniformly over a class of joint distributions of the response and the covariate which is rich enough for many practical applications furthermore to achieve optimality we show that the convergence rates of all possible estimators have a lower bound possessed by the kernel estimators

finite sample approximation results for principal component analysis a matrix perturbation approach principal component analysis pca is a standard tool for dimensional reduction of a set of n observations samples each with p variables in this paper using a matrix perturbation approach we study the nonasymptotic relation between the eigenvalues and eigenvectors of pca computed on a finite sample of size n and those of the limiting population pca as n oo as in machine learning we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample pca and population pca under a spiked covariance model in addition we also consider the relation between finite sample pca and the asymptotic results in the joint limit p n with p n c we present a matrix perturbation view of the phase transition phenomenon and a simple linear algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit moreover our analysis also applies for finite p n where we show that although there is no sharp phase transition as in the infinite case either as a function of noise level or as a function of sample size n the eigenvector of sample pca may exhibit a sharp loss of tracking suddenly losing its relation to the true eigenvector of the population pca matrix this occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise whose eigenvector points in a random direction

on subdifferential calculus for set valued mappings and optimality conditions abstract the aim of this paper is to establish formulas for the subdifferentials of the sum and the composition of convex set valued mappings under the attouch brezis qualification condition an application to a general set valued optimization problem is considered

compressed gaussian process for manifold regression nonparametric regression for large numbers of features p is an increasingly important problem if the sample size n is massive a common strategy is to partition the feature space and then separately apply simple models to each partition set this is not ideal when n is modest relative to p and we propose an alternative approach relying on random compression of the feature vector combined with gaussian process regression the proposed approach is particularly motivated by the setting in which the response is conditionally independent of the features given the projection to a low dimensional manifold conditionally on the random compression matrix and a smoothness parameter the posterior distribution for the regression surface and posterior predictive distributions are available analytically running the analysis in parallel for many random compression matrices and smoothness parameters model averaging is used to combine the results the algorithm can be implemented rapidly even in very large p and moderately large n nonparametric regression has strong theoretical justification and is found to yield state of the art predictive performance

manifold regularization and semi supervised learning some theoretical analyses manifold regularization belkin et al 2006 is a geometrically motivated framework for machine learning within which several semi supervised algorithms have been constructed here we try to provide some theoretical understanding of this approach our main result is to expose the natural structure of a class of problems on which manifold regularization methods are helpful we show that for such problems no supervised learner can learn effectively on the other hand a manifold based learner that knows the manifold or learns it from unlabeled examples can learn with relatively few labeled examples our analysis follows a minimax style with an emphasis on finite sample results in terms of n the number of labeled examples these results allow us to properly interpret manifold regularization and related spectral and geometric algorithms in terms of their potential use in semi supervised learning

computable representations for convex hulls of low dimensional quadratic forms let mathcal c be the convex hull of points 1 choose x 1 choose x t x in mathcal f subset re n representing or approximating mathcal c is a fundamental problem for global optimization algorithms based on convex relaxations of products of variables we show that if n 4 and mathcal f is a simplex then mathcal c has a computable representation in terms of matrices x that are doubly nonnegative positive semidefinite and componentwise nonnegative we also prove that if n 2 and mathcal f is a box then mathcal c has a representation that combines semidefiniteness with constraints on product terms obtained from the reformulation linearization technique rlt the simplex result generalizes known representations for the convex hull of x 1 x 2 x 1x 2 x in mathcal f when mathcal f subset re 2 is a triangle while the result for box constraints generalizes the well known fact that in this case the rlt constraints generate the convex hull of x 1 x 2 x 1x 2 x in mathcal f when n 3 and mathcal f is a box we show that a representation for mathcal c can be obtained by utilizing the simplex result for n 4 in conjunction with a triangulation of the 3 cube

metric regularity of epigraphical multivalued mappings and applications to vector optimization in this work we combine in a meaningful way two techniques of variational analysis and nonsmooth optimization on one hand we use the error bound approach to study the metric regularity of some special types of multifunctions and on the other hand we exploit the incompatibility between the metric regularity and the pareto minimality this method allows us to present some varepsilon fermat rules for set valued optimization problem in the setting of general banach spaces our results are comparable to several recent results in literature

lagrange necessary conditions for pareto minimizers in asplund spaces and applications abstract in this paper new necessary conditions for pareto minimal points to sets and pareto minimizers for constrained multiobjective optimization problems are established without the sequentially normal compactness property and the asymptotical compactness condition imposed on closed and convex ordering cones in bao and mordukhovich 10 and durea and dutta 5 respectively our approach is based on a version of the separation theorem for nonconvex sets and the subdifferentials of vector valued and set valued mappings furthermore applications in mathematical finance and approximation theory are discussed

the complexity of approximating a nonlinear program we consider the problem of finding the maximum of a multivariate polynomial inside a convex polytope we show that there is no polynomial time approximation algorithm for this problem even one with a very poor guarantee unless p np we show that even when the polynomial is quadratic i e quadratic programming there is no polynomial time approximation unless np is contained in quasi polynomial time