lipschitz operator ideals and the approximation property abstract we establish the basics of the theory of lipschitz operator ideals with the aim of recovering several classes of lipschitz maps related to absolute summability that have been introduced in the literature in the last years as an application we extend the notion and main results on the approximation property for banach spaces to the case of metric spaces

lipschitz p integral operators and lipschitz p nuclear operators national natural science foundation of china 10526034 10701063 nsf dms 0800061 dms 1068838 dms 1200370

emerging notions of norm attainment for lipschitz maps between banach spaces abstract we classify several notions of norm attaining lipschitz maps which were introduced previously and present the relations among them in order to verify proper inclusions we also analyze some results for the sets of lipschitz maps satisfying each of these properties to be dense or not in lip 0 x y for instance we characterize a banach space y with the radon nikod m property in terms of the denseness of norm attaining lipschitz maps with values in y further we introduce a property called the local directional bishop phelps bollobas property for lipschitz compact maps which extends the one studied previously for scalar valued functions and provide some new positive results

lipschitz p r s integral operators and lipschitz p r s nuclear operators abstract we introduce the notions of strongly lipschitz p r s nuclear operators and strongly lipschitz p r s integral operators we develop a theory of strongly lipschitz p r s nuclear operators and strongly lipschitz p r s integral operators which closely parallels the theory for linear operators their close relationships with other lipschitz operator ideals are studied

limited stochastic meta descent for kernel based online learning to improve the single run performance of online learning and reinforce its stability we consider online learning with limited adaptive learning rate in this letter the letter extends convergence proofs for norma to a range of step sizes then employs support vector learning with stochastic meta descent svmd limited to that range for step size adaptation so as to obtain an online kernel algorithm that combines theoretical convergence guarantees with good practical performance experiments on different data sets corroborate theoretical results well and show that our method is another promising way for online learning

model based online learning with kernels new optimization models and algorithms for online learning with kernels olk in classification regression and novelty detection are proposed in a reproducing kernel hilbert space unlike the stochastic gradient descent algorithm called the naive online reg minimization algorithm norma olk algorithms are obtained by solving a constrained optimization problem based on the proposed models by exploiting the techniques of the lagrange dual problem like vapnik s support vector machine svm the solution of the optimization problem can be obtained iteratively and the iteration process is similar to that of the norma this further strengthens the foundation of olk and enriches the research area of svm we also apply the obtained olk algorithms to problems in classification regression and novelty detection including real time background substraction to show their effectiveness it is illustrated that based on the experimental results of both classification and regression the accuracy of olk algorithms is comparable with traditional svm based algorithms such as svm and least square svm ls svm and with the state ofthe art algorithms such as kernel recursive least square krls method and projectron method while it is slightly higher than that of norma on the other hand the computational cost of the olk algorithm is comparable with or slightly lower than existing online methods such as above mentioned norma krls and projectron methods but much lower than that of svm based algorithms in addition different from svm and ls svm it is possible for olk algorithms to be applied to non stationary problems also the applicability of olk in novelty detection is illustrated by simulation results

fast iterative kernel principal component analysis we develop gain adaptation methods that improve convergence of the kernel hebbian algorithm kha for iterative kernel pca kim et al 2005 kha has a scalar gain parameter which is either held constant or decreased according to a predetermined annealing schedule leading to slow convergence we accelerate it by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector an additional normalization term then allows us to eliminate a tuning parameter in the annealing schedule finally we derive and apply stochastic meta descent smd gain vector adaptation schraudolph 1999 2002 in reproducing kernel hilbert space to further speed up convergence experimental results on kernel pca and spectral clustering of usps digits motion capture and image denoising and image super resolution tasks confirm that our methods converge substantially faster than conventional kha to demonstrate scalability we perform kernel pca on the entire mnist data set

a remark on the approximation of p compact operators by finite rank operators abstract the banach operator ideal k p of p compact operators was introduced in p d sinha a k karn compact operators whose adjoints factor through subspaces of l p studia math 150 2002 17 33 let p 1 and let x be a closed subspace of an l p space we show that x has the approximation property if and only if for every banach space y the linear space f y x of finite rank operators is k p norm dense in k p y x i e x has the k p approximation property

unconditional almost squareness and applications to spaces of lipschitz functions abstract we introduce an unconditional concept of almost squareness in order to provide a partial negative answer to the problem of existence of almost square dual banach spaces we also take advantage of this notion to provide some criteria of non duality of some subspaces of scalar as well as vector valued lipschitz functions

exponentiated gradient versus gradient descent for linear predictors we consider two algorithm for on line prediction based on a linear model the algorithms are the well known gradient descent gd algorithm and a new algorithm which we call eg they both maintain a weight vector using simple updates for the gd algorithm the update is based on subtracting the gradient of the squared error made on a prediction the eg algorithm uses the components of the gradient in the exponents of factors that are used in updating the weight vector multiplicatively we present worst case loss bounds for eg and compare them to previously known bounds for the gd algorithm the bounds suggest that the losses of the algorithms are in general incomparable but eg has a much smaller loss if only a few components of the input are relevant for the predictions we have performed experiments which show that our worst case upper bounds are quite tight already on simple artificial data

two lipschitz operator ideals abstract we introduce and investigate the concept of two lipschitz operator ideal between pointed metric spaces and banach spaces we show the basics of this new theory and we give a procedure to create a two lipschitz operator ideal from a linear operator ideal we apply our result to the ideals of strongly p summing and compact linear operator to obtain their corresponding two lipschitz operator ideal also we establish a natural relation between two lipschitz and bilinear maps and show that the two lipschitz factorable p dominated operators are those which are associated to the well known p semi integral bilinear operators

the dual space of l x y τp and the p approximation property we establish a representation of the dual space of l x y the space of bounded linear operators from a banach space x into a banach space y endowed with the topology p of uniform convergence on p compact subsets of x we apply this representation and solve the duality problem for the p approximation property p ap that is if the dual space x has the p ap then so does x however the converse does not hold in general we show that given 2 2p p 2 this subspace is the davie space in lq davie 1973 5 which does not have the approximation property it follows that for every 2 p there exists a banach space yp such that it has the p ap but its dual space yp fails to have the p ap we study the relation of the p ap with the denseness of finite rank operators in the topology p finally we introduce the p compact approximation property p cap and show for every 2 p that the davie space in c0 fails to have the p cap and also that a variant of the willis space willis 1992 17 has the p cap but it fails to have the p ap

weak and quasi approximation properties in banach spaces in this paper we introduce weak versions the weak approximation property the bounded weak approximation property and the quasi approximation property of the approximation property and derive various characterizations of these properties and we show that if the dual of a banach space x has the weak approximation property respectively the bounded weak approximation property then x itself has the weak approximation property respectively the bounded weak approximation property also we observe that the bounded weak approximation property is closely related to the quasi approximation property

iterative kernel principal component analysis for image modeling in recent years kernel principal component analysis kpca has been suggested for various image processing tasks requiring an image model such as e g denoising or compression the original form of kpca however can be only applied to strongly restricted image classes due to the limited number of training examples that can be processed we therefore propose a new iterative method for performing kpca the kernel hebbian algorithm which iteratively estimates the kernel principal components with only linear order memory complexity in our experiments we compute models for complex image classes such as faces and natural images which require a large number of training examples the resulting image models are tested in single frame super resolution and denoising applications the kpca model is not specifically tailored to these tasks in fact the same model can be used in super resolution with variable input resolution or denoising with unknown noise characteristics in spite of this both super resolution and denoising performance are comparable to existing methods

a polynomial hutton theorem with applications abstract extending a classical linear result due to hutton to a nonlinear setting we prove that a continuous homogeneous polynomial between banach spaces can be approximated by finite rank polynomials if and only if its adjoint can be approximated by finite rank linear operators among other consequences we apply this result to generalize a classical result due to aron and schottenloher about the approximation property on spaces of polynomials and a recent result due to caliskan and rueda about the quasi approximation property on projective symmetric tensor products

duality for lipschitz p summing operators abstract building upon the ideas of r arens and j eells 1956 1 we introduce the concept of spaces of banach space valued molecules whose duals can be naturally identified with spaces of operators between a metric space and a banach space on these spaces we define analogues of the tensor norms of chevet 1969 3 and saphar 1970 14 whose duals are spaces of lipschitz p summing operators in particular we identify the dual of the space of lipschitz p summing operators from a finite metric space to a banach space answering a question of j farmer and w b johnson 2009 6 and use it to give a new characterization of the non linear concept of lipschitz p summing operator between metric spaces in terms of linear operators between certain banach spaces more generally we define analogues of the norms of j t lapreste 1976 11 whose duals are analogues of a pietsch s p r s summing operators a pietsch 1980 12 as a special case we get a lipschitz version of q p dominated operators

fixed points of uniformly lipschitzian mappings in spaces with uniformly normal structure dans un espace de banach x on etudie quand reflexivite et structure normale sont suffisantes pour assurer pour de bon 1 la propriete de point fixe pour des auto applications uniformement lipschitziennes

linearization of bounded holomorphic mappings on banach spaces the main result in this paper is the following linearization theorem for each open set u in a complex banach space e there is a complex banach space goo u and a bounded holomorphic mapping gu u g u with the following universal property for each complex banach space f and each bounded holomorphic mapping f u f there is a unique continuous linear operator tf goo u f such that tf gu f the correspondence f tf is an isometric isomorphism between the space ho u f of all bounded holomorphic mappings from u into f and the space l g u f of all continuous linear operators from g u into f these properties characterize g u uniquely up to an isometric isomorphism the rest of the paper is devoted to the study of some aspects of the interplay between the spaces h o u f and l g u f this paper consists of five sections in 1 we establish our notation and terminology in 2 we prove the aforementioned linearization theorem in 3 we translate certain properties of a mapping f e h u f into properties of the corresponding operator tf e l g u f we show for instance that f has a relatively compact range if and only if tf is a compact operator in 4 we give a seminorm characterization of the unique locally convex topology t y on h u f such that the correspondence f tf is a topological isomorphism between the spaces h u f ty and l g u f t where tc denotes the compact open topology finally in 5 we use the preceding results to establish necessary and sufficient conditions for the spaces g u and h u to have the approximation property these are holomorphic analogues of classical results of a grothendieck 8 and complement results of r aron and m schottenloher 2 we show in particular that if u is a balanced bounded open set in a complex banach space e then g u has the approximation property if and only if e has the approximation property we also show that if u is an arbitrary open set in a complex banach space e then h u has the approximation property if and only if for each complex banach space f each mapping in h u f with a relatively compact range can be uniformly approximated on u by mappings in h u f with finite dimensional range since it is still unknown whether received by the editors april 10 1989 1980 mathematics subject classification 1985 revision primary 46g20 46e15 secondary 46e10 d 1991 american mathematical society 0002 9947 91 1 00 25 per page

step size adaptation in reproducing kernel hilbert space this paper presents an online support vector machine svm that uses the stochastic meta descent smd algorithm to adapt its step size automatically we formulate the online learning problem as a stochastic gradient descent in reproducing kernel hilbert space rkhs and translate smd to the nonparametric setting where its gradient trace parameter is no longer a coefficient vector but an element of the rkhs we derive efficient updates that allow us to perform the step size adaptation in linear time we apply the online svm framework to a variety of loss functions and in particular show how to handle structured output spaces and achieve efficient online multiclass classification experiments show that our algorithm outperforms more primitive methods for setting the gradient step size

weak strong continuity of multilinear mappings and the pelczynski pitt theorem abstract in generalizing a result of pelczynski the sequential weak to norm continuity ofn linear mappings between certain banach spaces including spaces of type and cotype will be studied in particular it is shown that everyn linear continuous mapping from lp1 lpninto lqis compact if and only if i 1n 1 pi

galois connection between lipschitz and linear operator ideals and minimal lipschitz operator ideals abstract we establish a relation between lipschitz operator ideals and linear operator ideals which fits in the framework of galois connection between lattices we use this relationship to give a criterion which allows us to recognize when a banach lipschitz operator ideal is of composition type or not also we introduce the concept of minimal banach lipschitz operator ideal which have analogous properties to minimal banach operator ideals we characterize minimal banach lipschitz operator ideals which are of composition type and present examples which are not of this class

normal structure coefficients of l p ω let x be a uniformly convex banach space and n x the normal structure coefficient of x in this paper it is proved that n x can be calculated by considering only sets whose points are equidistant from their chebyshev centre this result is applied to prove that n l p min 2 1 1 p 2 1 p being a finite measure space the computation of n l p lets us also calculate some other coefficients related to the normal structure

worst case quadratic loss bounds for prediction using linear functions and gradient descent studies the performance of gradient descent gd when applied to the problem of online linear prediction in arbitrary inner product spaces we prove worst case bounds on the sum of the squared prediction errors under various assumptions concerning the amount of a priori information about the sequence to predict the algorithms we use are variants and extensions of online gd whereas our algorithms always predict using linear functions as hypotheses none of our results requires the data to be linearly related in fact the bounds proved on the total prediction loss are typically expressed as a function of the total loss of the best fixed linear predictor with bounded norm all the upper bounds are tight to within constants matching lower bounds are provided in some cases finally we apply our results to the problem of online prediction for classes of smooth functions

new support vector algorithms we propose a new class of support vector algorithms for regression and classification in these algorithms a parameter lets one effectively control the number of support vectors while this can be useful in its own right the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm the accuracy parameter epsilon in the regression case and the regularization constant c in the classification case we describe the algorithms give some theoretical results concerning the meaning and the choice of and report experimental results

a note on the unification of adaptive online learning in online convex optimization adaptive algorithms which can utilize the second order information of the loss function s sub gradient have shown improvements over standard gradient methods this paper presents a framework follow the bregman divergence leader that unifies various existing adaptive algorithms from which new insights are revealed under the proposed framework two simple adaptive online algorithms with improvable performance guarantee are derived furthermore a general equation derived from a matrix analysis generalizes the adaptive learning to nonlinear case with kernel trick

on p compact mappings and the p approximation property abstract the notion of p compact sets arises naturally from grothendieck s characterization of compact sets as those contained in the convex hull of a norm null sequence the definition due to sinha and karn 2002 leads to the concepts of p approximation property and p compact operators which form an ideal with its ideal norm p this paper examines the interaction between the p approximation property and certain space of holomorphic functions the p compact analytic functions in order to understand these functions we define a p compact radius of convergence which allows us to give a characterization of the functions in the class we show that p compact holomorphic functions behave more like nuclear than compact maps we use the product of schwartz to characterize the p approximation property of a banach space in terms of p compact homogeneous polynomials and in terms of p compact holomorphic functions with range on the space finally we show that p compact holomorphic functions fit into the framework of holomorphy types which allows us to inspect the p approximation property our approach also allows us to solve several questions posed by aron maestre and rueda 2010

fast curvature matrix vector products for second order gradient descent we propose a generic method for iteratively approximating various second order gradient steps newton gauss newton levenberg marquardt and natural gradient in linear time per iteration using special curvature matrix vector products that can be computed in o n two recent acceleration techniques for on line learning matrix momentum and stochastic meta descent smd implement this approach since both were originally derived by very different routes this offers fresh insight into their operation resulting in further improvements to smd

an lp inequality and its applications to fixed point theory and approximation theory on etablit une inegalite l p pour 1 p 2 et on donne des applications aux points fixes des applications uniformement lipschitziennes et aux meilleures approximations fortement uniques

density of finite rank operators in the banach space of p compact operators a banach space x is said to have the kp approximation property kp ap if for every banach space y the space f y x of finite rank operators is dense in the space kp y x of p compact operators endowed with its natural ideal norm kp in this paper we study this notion that has been previously treated by sinha and karn 2002 in 15 as application the kp ap of dual banach spaces is characterized via density of finite rank operators in the space of quasi p nuclear operators for the p summing norm this allows to obtain a relation between the kp ap and saphar s approximation property as another application the kp ap is characterized in terms of a trace condition finally we relate the kp ap to the p p approximation property introduced in sinha and karn 2002 15 for subspaces of lp spaces

approximation properties and schauder decompositions in lipschitz free spaces abstract we prove that the lipschitz free space over a doubling metric space has the bounded approximation property we also show that the lipschitz free spaces over l 1 n or l 1 have monotone finite dimensional schauder decompositions

lipschitz compact operators abstract we introduce the notion of lipschitz compact weakly compact finite rank approximable operators from a pointed metric space x into a banach space e we prove that every strongly lipschitz p nuclear operator is lipschitz compact and every strongly lipschitz p integral operator is lipschitz weakly compact a theory of lipschitz compact weakly compact finite rank operators which closely parallels the theory for linear operators is developed in terms of the lipschitz transpose map of a lipschitz operator we state lipschitz versions of schauder type theorems on the weak compactness of the adjoint of a weakly compact linear operator

the p approximation property in terms of density of finite rank operators abstract we characterize the p approximation property p ap introduced by sinha and karn d p sinha a k karn compact operators whose adjoints factor through subspaces of l p studia math 150 2002 17 33 in terms of density of finite rank operators in the spaces of p compact and of adjoints of p summable operators as application the p ap of dual banach spaces is characterized via density of finite rank operators in the space of quasi p nuclear operators this relates the p ap to saphar s approximation property ap p as another application the p ap is characterized via a trace condition allowing to define the trace functional on certain subspaces of the space of nuclear operators

online learning with kernels kernel based algorithms such as support vector machines have achieved considerable success in various problems in batch setting where all of the training data is available in advance support vector machines combine the so called kernel trick with the large margin idea there has been little use of these methods in an online setting suitable for real time applications in this paper we consider online learning in a reproducing kernel hilbert space by considering classical stochastic gradient descent within a feature space and the use of some straightforward tricks we develop simple and computationally efficient algorithms for a wide range of problems such as classification regression and novelty detection in addition to allowing the exploitation of the kernel trick in an online setting we examine the value of large margins for classification in the online setting with a drifting target we derive worst case loss bounds and moreover we show the convergence of the hypothesis to the minimizer of the regularized risk functional we present some experimental results that support the theory as well as illustrating the power of the new algorithms for online novelty detection

some properties of lipschitz strongly p summing operators abstract we consider the space of molecules endowed with the transposed version of the chevet saphar norm and we identify its dual space with the space of lipschitz strongly p summing operators we also extend some old results to the category of lipschitz mappings and we give a factorization result of lipschitz p r s summing operators

the bishop phelps bollobas property for lipschitz maps abstract in this paper we introduce and study a lipschitz version of the bishop phelps bollobas property lip bpb property this property deals with the possibility of making a uniformly simultaneous approximation of a lipschitz map f and a pair of points at which f almost attains its norm by a lipschitz map g and a pair of points such that g strongly attains its norm at the new pair of points we first show that if m is a finite pointed metric space and y is a finite dimensional banach space then the pair m y has the lip bpb property and that both finiteness assumptions are needed next we show that if m is a uniformly gromov concave pointed metric space i e the molecules of m form a set of uniformly strongly exposed points then m y has the lip bpb property for every banach space y we further prove that this is the case for finite concave metric spaces ultrametric spaces and holder metric spaces the extension of the lip bpb property from m r to some banach spaces y and some results for compact lipschitz maps are also discussed

the banach ideal of a compact operators and related approximation properties abstract we use the notion of a compact sets determined by an operator ideal a introduced by carl and stephani 1984 to show that many known results of certain approximation properties and several ideals of compact operators can be systematically studied under this framework for banach operator ideals a we introduce a way to measure the size of a compact sets and use it to give a norm on k a the ideal of a compact operators then we study two types of approximation properties determined by a compact sets we focus our attention on an approximation property which makes use of the norm defined on k a this notion fits the definition of the a approximation property recently introduced by oja 2012 with k a instead of a we exemplify the power of the carl stephani theory and the geometric structure introduced here by appealing to some recent developments on p compactness