regression on manifolds estimation of the exterior derivative collinearity and near collinearity of predictors cause difficulties when doing regression in these cases variable selection becomes untenable because of mathematical issues concerning the existence and numerical stability of the regression coefficients and interpretation of the coefficients is ambiguous because gradients are not defined using a differential geometric interpretation in which the regression coefficients are interpreted as estimates of the exterior derivative of a function we develop a new method to do regression in the presence of collinearities our regularization scheme can improve estimation error and it can be easily modified to include lasso type regularization these estimators also have simple extensions to the large p small n context

large scale multiple kernel learning while classical kernel based learning algorithms are based on a single kernel in practice it is often desirable to use multiple kernels lanckriet et al 2004 considered conic combinations of kernel matrices for classification leading to a convex quadratically constrained quadratic program we show that it can be rewritten as a semi infinite linear program that can be efficiently solved by recycling the standard svm implementations moreover we generalize the formulation and our method to a larger class of problems including regression and one class classification experimental results show that the proposed algorithm works for hundred thousands of examples or hundreds of kernels to be combined and helps for automatic model selection improving the interpretability of the learning result in a second part we discuss general speed up mechanism for svms especially when used with sparse feature maps as appear for string kernels allowing us to train a string kernel svm on a 10 million real world splice data set from computational biology we integrated multiple kernel learning in our machine learning toolbox shogun for which the source code is publicly available at http www fml tuebingen mpg de raetsch projects shogun

optimal birth control for an age dependent n dimensional food chain model abstract in this paper we investigate optimal policies for an age dependent n dimensional food chain model which is controlled by fertility by using dubovitskii milyutin s general theory the maximum principles are obtained for problems with free terminal states infinite horizon and target sets respectively

smoothing splines regression derivatives and deconvolution abstract the statistical properties of a cubic smoothing spline and its derivative are analyzed it is shown that unless unnatural boundary conditions hold the integrated squared bias is dominated by local effects near the boundary similar effects are shown to occur in the regularized solution of a translation kernel integral equation these results are derived by developing a fourier representation for a smoothing spline author

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

generalization error bounds in semi supervised classification under the cluster assumption we consider semi supervised classification when part of the available data is unlabeled these unlabeled data can be useful for the classification problem when we make an assumption relating the behavior of the regression function to that of the marginal distribution seeger 2000 proposed the well known cluster assumption as a reasonable one we propose a mathematical formulation of this assumption and a method based on density level sets estimation that takes advantage of it to achieve fast rates of convergence both in the number of unlabeled examples and the number of labeled examples

spline smoothing the equivalent variable kernel method on considere une approche spline pour la regression non parametrique et l estimation de courbe on montre que dans un certain sens le lissage spline correspond approximativement au lissage par une methode du noyau avec une largeur de bande dependant de la densite locale des points du plan de regression

use of the zero norm with linear models and kernel methods we explore the use of the so called zero norm of the parameters of linear models in learning minimization of such a quantity has many uses in a machine learning context for variable or feature selection minimizing training error and ensuring sparsity in solutions we derive a simple but practical method for achieving these goals and discuss its relationship to existing techniques of minimizing the zero norm the method boils down to implementing a simple modification of vanilla svm namely via an iterative multiplicative rescaling of the training data applications we investigate which aid our discussion include variable and feature selection on biological microarray data and multicategory classification

semi supervised learning using greedy max cut graph based semi supervised learning ssl methods play an increasingly important role in practical machine learning systems particularly in agnostic settings when no parametric information or other prior knowledge is available about the data distribution given the constructed graph represented by a weight matrix transductive inference is used to propagate known labels to predict the values of all unlabeled vertices designing a robust label diffusion algorithm for such graphs is a widely studied problem and various methods have recently been suggested many of these can be formalized as regularized function estimation through the minimization of a quadratic cost however most existing label diffusion methods minimize a univariate cost with the classification function as the only variable of interest since the observed labels seed the diffusion process such univariate frameworks are extremely sensitive to the initial label choice and any label noise to alleviate the dependency on the initial observed labels this article proposes a bivariate formulation for graph based ssl where both the binary label information and a continuous classification function are arguments of the optimization this bivariate formulation is shown to be equivalent to a linearly constrained max cut problem finally an efficient solution via greedy gradient max cut ggmc is derived which gradually assigns unlabeled vertices to each class with minimum connectivity once convergence guarantees are established this greedy max cut based ssl is applied on both artificial and standard benchmark data sets where it obtains superior classification accuracy compared to existing state of the art ssl methods moreover ggmc shows robustness with respect to the graph construction method and maintains high accuracy over extensive experiments with various edge linking and weighting schemes

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

remarks on functional canonical variates alternating least squares methods and ace we discuss properties of some data analytic methods which are intimately related to each other alternating least squares als correspondence analysis and more recently breiman and friedman s ace algorithm the application of these methods to regression produces nonparametric estimators of nonlinear transformations both of the response and the predictors we point out some anomalies as well as some curiosities in the mathematics of these methods and we relate them to some areas in computer aided tomography projection pursuit regression and nonlinear devices in the theory of noise

generalized additive models some applications abstract generalized additive models have the form x fj x j where might be the regression function in a multiple regression or the logistic transformation of the posterior probability pr y 1 x in a logistic regression in fact these models generalize the whole family of generalized linear models x x where x g x is some transformation of the regression function we use the local scoring algorithm to estimate the functions fj xj nonparametrically using a scatterplot smoother as a building block we demonstrate the models in two different analyses a nonparametric analysis of covariance and a logistic regression the procedure can be used as a diagnostic tool for identifying parametric transformations of the covariates in a standard linear analysis a variety of inferential tools have been developed to aid the analyst in assessing the relevance and significance of the estimated functions these include confidence curves degrees of freedom estimates and approximat

multiple graph regularized graph transduction via greedy gradient max cut abstract graph transduction methods have been widely adopted for label prediction under semi supervised settings to alleviate the relevant sensitivity to initial labels and graph construction processes recent studies have been aiming at developing robust graph transduction techniques in particular the graph transduction method via greedy gradient max cut ggmc that minimizes a cost function over a continuous classification function and a binary label variable has been successfully applied to a wide range of applications however this method predominately relies on the choice of a high quality single graph representation often leading to unstable performance due to selection bias to tackle this major drawback we leverage an ensemble learning framework into the ggmc method for exploiting the advantage of constructing and combining multiple graphs as opposed to performing constrained max cut on a single graph the proposed multiple graph greedy gradient max cut method mg ggmc simultaneously solves the label prediction and the true graph estimation problems specifically the true graph is approximated by a linear combination of a set of constructed graphs the coefficients of the linear combination are learned automatically by alternately minimizing a unified objective function in an iterative manner comparison studies with representative methods across various real world benchmarks conspicuously demonstrate the efficaciousness and the superiority of the proposed algorithm in standard evaluation metrics

automatic smoothing of regression functions in generalized linear models abstract we consider the penalized likelihood method for estimating nonparametric regression functions in generalized linear models nelder and wedderburn 1972 and present a generalized cross validation procedure for empirically assessing an appropriate amount of smoothing in these estimates asymptotic arguments and numerical simulations are used to show that the generalized cross validatory procedure preforms well from the point of view of a weighted mean squared error criterion the methodology adds to the battery of graphical tools for model building and checking within the generalized linear model framework included are two examples motivated by medical and horticultural applications

linear smoothers and additive models we study linear smoothers and their use in building nonparametric regression models in the first part of this paper we examine certain aspects of linear smoothers for scatterplots examples of these are the running mean and running line kernel and cubic spline smoothers the eigenvalue and singular value decompositions of the corresponding smoother matrix are used to describe qualitatively a smoother and several other topics such as the number of degrees of freedom of a smoother are discussed in the second part of the paper we describe how linear smoothers can be used to estimate the additive model a powerful nonparametric regression model using the backfitting algorithm we show that backfitting is the gauss seidel iterative method for solving a set of normal equations associated with the additive model we provide conditions for consistency and nondegeneracy and prove convergence for the backfitting and related algorithms for a class of smoothers that includes cubic spline smoothers

fundamental sampling patterns for low rank multi view data completion abstract we consider the multi view data completion problem i e to complete a matrix u u 1 u 2 where the ranks of u u1 and u2 are given in particular we investigate the fundamental conditions on the sampling pattern i e locations of the sampled entries for finite completability of such a multi view data given the corresponding rank constraints we provide a geometric analysis on the manifold structure for multi view data to incorporate more than one rank constraint we derive a probabilistic condition in terms of the number of samples per column that guarantees finite completability with high probability finally we derive the guarantees for unique completability numerical results demonstrate reduced sampling complexity when the multi view structure is taken into account as compared to when only low rank structure of individual views is taken into account then we propose an apporach using newton s method to almost achieve these information theoretic bounds for mulit view data retrieval by taking advantage of the rank decomposition and the analysis in this work

the performance of some manufacturing queueing networks when some of the machines are in the failure mode abstract using the theory of right continuous markov processes with values in n k an optimal control problem concerning the performance of manufacturing queueing networks when some of the machines are in the failure mode is dealt with

optimal control for age structured population dynamics of incomplete data we are concerned with the control question for linear age structured population dynamics of incomplete initial data more precisely the initial population age distribution is supposed to be unknown we here generalize the notion of no regret control of lions 1992 10 to such singular population dynamics following the method by nakoulima omrane and velin 2000 16 we prove that the problem we are considering has a unique no regret control that we characterize by a singular optimality system

optimal control of averaged state of a parabolic equation with missing boundary condition we consider the optimal control of general heat governed by an operator depending on an unknown parameter and with missing boundary condition using the notion of no regret and low regret control

exploring the nature of covariate effects in the proportional hazards model we discuss an exploratory technique for investigating the nature of covariate effects in cox s proportional hazards model this technique features an additive term sigma p1 fj chi ij in place of the usual linear term sigma p1 chi ij beta j where chi i1 chi i2 chi ip are covariate values for the ith individual the fj are unspecified smooth functions that are estimated using scatterplot smoothers these functions can be used for descriptive purposes or to suggest transformations of the covariates the estimation technique is a variation of the local scoring algorithm for generalized additive models hastie and tibshirani 1986 statistical science 1 297 318

fitting a bivariate additive model by local polynomial regression while the additive model is a popular nonparametric regression method many of its theoretical properties are not well understood especially when the backfitting algorithm is used for computation of the estimators this article explores those properties when the additive model is fitted by local polynomial regression sufficient conditions guaranteeing the asymptotic existence of unique estimators for the bivariate additive model are given asymptotic approximations to the bias and the variance of a homoscedastic bivariate additive model with local polynomial terms of odd and even degree are computed this model is shown to have the same rate of convergence as that of univariate local polynomial regression

prediction intervals for neural networks via nonlinear regression standard methods for computing prediction intervals in nonlinear regression can be effectively applied to neural networks when the number of training points is large simulations show however that these methods can generate unreliable prediction intervals on smaller datasets when the network is trained to convergence stopping the training algorithm prior to convergence to avoid overfitting reduces the effective number of parameters but can lead to prediction intervals that are too wide we present an alternative approach to estimating prediction intervals using weight decay to fit the network and show via a simulation study that this method may be effective in overcoming some of the shortcomings of the other approaches

optimal birth control of population dynamics abstract we study optimal birth control policies for an age structured population of mckendrick type which is a distributed parameter system involving first order partial differential equations with nonlocal bilinear boundary control the functional analytic approach of dubovitskii and milyutin is adopted in the investigation maximum principles for problems with a free end condition a fixed final horizon are developed and the time optimal control problem the problem with target sets and the infinite planning horizon case are investigated

flexible parsimonious smoothing and additive modeling a simple method is presented for fitting regression models that are nonlinear in the explanatory variables despite its simplicity or perhaps because of it the method has some powerful characteristics that cause it to be competitive with and often superior to more sophisticated techniques especially for small data sets in the presence of high noise

semiparametric estimates of the relation between weather and electricity sales abstract a nonlinear relationship between electricity sales and temperature is estimated using a semiparametric regression procedure that easily allows linear transformations of the data this accommodates introduction of covariates timing adjustments due to the actual billing schedules and serial correlation the procedure is an extension of smoothing splines with the smoothness parameter estimated from minimization of the generalized cross validation criterion introduced by craven and wahba 1979 estimates are presented for residential sales for four electric utilities and are compared with models that represent the weather using only heating and cooling degree days or with piecewise linear splines

on data integration problems with manifolds this article focuses on data integration problems where the predictor variables for some response variable partition into known subsets this type of data is often referred to as multi view data a

bayesian confidence intervals for the cross validated smoothing spline summary we consider the model y t g ti ei i 17 2 n where g t t 0 1 is a smooth function and the ei are independent n 0 a2 errors with g2 unknown the cross validated smoothing spline can be used to estimate g non parametrically from

joint harmonic functions and their supervised connections the cluster assumption had a significant impact on the reasoning behind semi supervised classification methods in graph based learning the literature includes numerous applications where harmonic functions provided estimates that conformed to data satisfying this well known assumption but the relationship between this assumption and harmonic functions is not as well understood theoretically we investigate these matters from the perspective of supervised kernel classification and provide concrete answers to two fundamental questions i under what conditions do semi supervised harmonic approaches satisfy this assumption ii if such an assumption is satisfied then why precisely would an observation sacrifice its own supervised estimate in favor of the cluster first a harmonic function is guaranteed to assign labels to data in harmony with the cluster assumption if a specific condition on the boundary of the harmonic function is satisfied second it is shown that any harmonic function estimate within the interior is a probability weighted average of supervised estimates where the weight is focused on supervised kernel estimates near labeled cases we demonstrate that the uniqueness criterion for harmonic estimators is sensitive when the graph is sparse or the size of the boundary is relatively small this sets the stage for a third contribution a new regularized joint harmonic function for semi supervised learning based on a joint optimization criterion mathematical properties of this estimator such as its uniqueness even when the graph is sparse or the size of the boundary is relatively small are proven a main selling point is its ability to operate in circumstances where the cluster assumption may not be fully satisfied on real data by compromising between the purely harmonic and purely supervised estimators the competitive stature of the new regularized joint harmonic approach is established

local likelihood estimation abstract a scatterplot smoother is applied to data of the form x 1 y 1 x 2 y 2 xn yn and uses local fitting to estimate the dependence of y on x a simple example is the running lines smoother which fits a least squares line to the y values falling in a window around each x value the value of the estimated function at x is given by the value of the least squares line at x a smoother generalizes the least squares line which assumes that the dependence of y on x is linear in this article we extend the idea of local fitting to likelihood based regression models one such application is to the class of generalized linear models nelder and wedderburn 1972 we enlarge this class by replacing the covariate form 0 x 1 with an unspecified smooth function s x this function is estimated from the data by a technique we call local likelihood estimation the method consists of maximum likelihood estimation for 0 and 1 applied in a window around each x value multiple covariates are incor

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

sparse multiple kernel learning for signal processing applications in many signal processing applications grouping of features during model development and the selection of a small number of relevant groups can be useful to improve the interpretability of the learned parameters while a lot of work based on linear models has been reported to solve this problem in the last few years multiple kernel learning has come up as a candidate to solve this problem in nonlinear models since all of the multiple kernel learning algorithms to date use convex primal problem formulations the kernel weights selected by these algorithms are not strictly the sparsest possible solution the main reason for using a convex primal formulation is that efficient implementations of kernel based methods invariably rely on solving the dual problem this work proposes the use of an additional log based concave penalty term in the primal problem to induce sparsity in terms of groups of parameters a generalized iterative learning algorithm which can be used with a linear combination of this concave penalty term with other penalty terms is given for model parameter estimation in the primal space it is then shown that a natural extension of the method to nonlinear models using the kernel trick results in a new algorithm called sparse multiple kernel learning smkl which generalizes group feature selection to kernel selection smkl is capable of exploiting existing efficient single kernel algorithms while providing a sparser solution in terms of the number of kernels used as compared to the existing multiple kernel learning framework a number of signal processing examples based on the use of mass spectra for cancer detection hyperspectral imagery for land cover classification and nir spectra from wheat fescue grass and diesel are given to highlight the ability of smkl to achieve a very high accuracy with a very few kernels

structural multiple empirical kernel learning multiple kernel learning mkl can boost classification performance through using multiple kernels rather than a single fixed one unlike the traditional mkl with the implicit kernels multiple empirical kernel learning mekl explicitly maps input data into multiple feature spaces this paper focuses on mekl and proposes an effective threefold structural mekl tsmekl the first fold structure is the space structural information between different mapped feature spaces the second one is the class discriminant information within each mapped feature space the third one is the cluster structural information of samples in each mapped feature space the classical mekl mainly pays attention to the first two structures but neglects the last one the proposed tsmekl introduces the cluster structural information into mekl doing so can simultaneously utilize the space the class and the cluster information in the way from globality to locality therefore tsmekl utilizes threefold structural information to result in the improvement of classification performance to the best of our knowledge it is the first time to introduce the cluster information into the mekl framework the main advantage of the developed tsmekl is considering different folds of data information to improve classification performance the experimental results validate the feasibility and effectiveness of tsmekl moreover we discuss the theoretical and experimental generalization risk bound of the proposed algorithm in terms of the rademacher complexity

a variational method for learning sparse and overcomplete representations an expectation maximization algorithm for learning sparse and overcomplete data representations is presented the proposed algorithm exploits a variational approximation to a range of heavy tailed distributions whose limit is the laplacian a rigorous lower bound on the sparse prior distribution is derived which enables the analytic marginalization of a lower bound on the data likelihood this lower bound enables the development of an expectation maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients

on the pareto control and no regret control for distributed systems with incomplete data we discuss the control of distributed systems with incomplete data following the notion of no regret control or equivalently pareto control used by lions in c r acad sci paris ser i math 302 1986 pp 223 227 and c r acad sci paris ser i math 302 1992 pp 1253 1257 we associate with the no regret control a sequence of low regret controls defined by a quadratic perturbation previously used by nakoulima omrane and velin in c r acad sci paris ser i math 330 2000 pp 801 806 in the first part we prove that the perturbed system corresponds to a sequence of standard control problems and converges to the no regret or pareto control for which we obtain a singular optimality system we give also some applications in the second part we show how the method can be extended to the evolution case equations of parabolic type petrowsky type or hyperbolic type are considered

regression by local fitting methods properties and computational algorithms abstract local regression is a procedure for estimating regression surfaces by the local fitting of linear or quadratic functions of the independent variables in a moving fashion that is analogous to how a moving average is computed for a time series the advantage of the methodology over the global fitting of parametric functions of the independent variables by least squares the current paradigm in regression studies is that a much wider class of regression functions can be estimated without distortion in this paper we discuss the methods their statistical properties and computational algorithms

density sensitive semisupervised inference semisupervised methods are techniques for using labeled data x 1 y 1 ldots x n y n together with unlabeled data x n 1 ldots x n to make predictions these methods invoke some assumptions that link the marginal distribution p x of x to the regression function f x for example it is common to assume that f is very smooth over high density regions of p x many of the methods are ad hoc and have been shown to work in specific examples but are lacking a theoretical foundation we provide a minimax framework for analyzing semisupervised methods in particular we study methods based on metrics that are sensitive to the distribution p x our model includes a parameter alpha that controls the strength of the semisupervised assumption we then use the data to adapt to alpha

an application of combinatorial optimization to statistical physics and circuit layout design we study the problem of finding ground states of spin glasses with exterior magnetic field and the problem of minimizing the number of vias holes on a printed circuit board or contacts on a chip subject to pin preassignments and layer preferences the former problem comes up in solid state physics and the latter in very large scale integrated vlsi circuit design and in printed circuit board design both problems can be reduced to the max cut problem in graphs based on a partial characterization of the cut polytope we design a cutting plane algorithm and report on computational experience with it our method has been used to solve max cut problems on graphs with up to 1 600 nodes

locally weighted regression an approach to regression analysis by local fitting abstract locally weighted regression or loess is a way of estimating a regression surface through a multivariate smoothing procedure fitting a function of the independent variables locally and in a moving fashion analogous to how a moving average is computed for a time series with local fitting we can estimate a much wider class of regression surfaces than with the usual classes of parametric functions such as polynomials the goal of this article is to show through applications how loess can be used for three purposes data exploration diagnostic checking of parametric models and providing a nonparametric regression surface along the way the following methodology is introduced a a multivariate smoothing procedure that is an extension of univariate locally weighted regression b statistical procedures that are analogous to those used in the least squares fitting of parametric functions c several graphical methods that are useful tools for understanding loess estimates and checking the a

large margin semi supervised learning in classification semi supervised learning occurs when a large amount of unlabeled data is available with only a small number of labeled data in such a situation how to enhance predictability of classification through unlabeled data is the focus in this article we introduce a novel large margin semi supervised learning methodology using grouping information from unlabeled data together with the concept of margins in a form of regularization controlling the interplay between labeled and unlabeled data based on this methodology we develop two specific machines involving support vector machines and learning denoted as ssvm and spsi through difference convex programming in addition we estimate the generalization error using both labeled and unlabeled data for tuning regularizers finally our theoretical and numerical analyses indicate that the proposed methodology achieves the desired objective of delivering high performance in generalization particularly against some strong performers

decentralized control of service rates in a closed jackson network the authors consider a closed jackson network with m nodes the service rate at each node is controllable in a decentralized manner i e it will be controlled based on local information extracted from that node only for each node there is a holding cost and an operating cost it is assumed that both costs are time homogeneous and that the operating cost is a linear function of the service rate however both costs are arbitrary functions of the number of jobs at the node the objective is to minimize the time average expected total cost it is shown that there exists an optimal control characterized by a set of thresholds one for each node such that it is optimal for each node to serve at zero rates if the number of jobs there is below or at the threshold and serve at maximum allowed rates when the number of jobs exceeds the threshold this structure of the optimal control is identified by using the product form solution of the jackson network and the duality theory of linear programming

on a population dynamics control problem with age dependence and spatial structure we consider the control problem for a population dynamics model with age dependence spatial structure and a nonlocal birth process arising as a boundary condition we examine the controllability at a given time t and show that approximate controllability holds for every fixed finite time t as a consequence a new uniqueness condition continuation result is proved

cost constraint based multiple kernel least squares support vector machine and its application considering the neglect of kernel function cost and lack of sparsity of multiple kernel least squares support vector machine mk lssvm a cost constraint based multiple kernel least squares vector machine with sparsity is pro posed the primal optimal problem of mk lssvm is converted into second order cone programming and then the weight of complex kernel function is restricted by introducing cost factors so as to save storage space and computing time of variable quantity furthermore the kernel matrices are reduced by schmidt orthogonalization theory to lower computational com plexity the total cost of multiple kernel learning can be evaluated according to the number of support vectors and active kernel functions the simulation results on testing datasets show that the proposed method can achieve the same accuracy as mk lssvm by using less support vectors and simpler mixture kernel functions with cheaper consumption and better real time performance the cost vaule of froth flotation mineral recovery prediction used the proposed method reduces 27 56

analysis of field experiments by least squares smoothing summary assuming a smooth trend plus independent error model for the environmental effects in the yields of a field plot experiment least squares smoothing methods are developed to estimate both the treatment effects and the unknown trend treatment estimates are closely related to those resulting from a generalized least squares analysis in which the covariance structure for the environmental effects has a particular form however the main emphases are on the accuracy of treatment estimates under a fixed smooth trend plus error model and the exploratory power of the basic method to isolate trend effects of unknown form although the detailed development is for the one dimensional case generalizations of the smoothness concept and extensions to two dimensions are also discussed application of the basic method is illustrated on three data sets and the results compared with other analyses

estimating semiparametric arch models by kernel smoothing methods we investigate a class of semiparametric arch models that includes as a special case the partially nonparametric pnp model introduced by engle and ng 1993 and which allows for both flexible dynamics and flexible function form with regard to the news impact function we show that the functional part of the model satisfies a type ii linear integral equation and give simple conditions under which there is a unique solution we propose an estimation method that is based on kernel smoothing and profiled likelihood we establish the distribution theory of the parametric components and the pointwise distribution of the nonparametric component of the model we also discuss efficiency of both the parametric part and the nonparametric part we investigate the performance of our procedures on simulated data and on a sample of s p500 index returns we find evidence of asymmetric news impact functions consistent with the parametric analysis

augmented partial residuals a simple way of encouraging a nonlinear effect in a regression model to show itself in a partial residual parres plot component plus residual plot is to include a single quadratic term in the regression it is computationally cheap to do this for each of the independent variables examples show that the resulting set of augmented partial residual plots gives insights that are not available from standard parres or added variable plots

on semi supervised linear regression in covariate shift problems semi supervised learning approaches are trained using the full training labeled data and available testing unlabeled data demonstrations of the value of training with unlabeled data typically depend on a smoothness assumption relating the conditional expectation to high density regions of the marginal distribution and an inherent missing completely at random assumption for the labeling so called covariate shift poses a challenge for many existing semi supervised or supervised learning techniques covariate shift models allow the marginal distributions of the labeled and unlabeled feature data to differ but the conditional distribution of the response given the feature data is the same an example of this occurs when a complete labeled data sample and then an unlabeled sample are obtained sequentially as it would likely follow that the distributions of the feature data are quite different between samples the value of using unlabeled data during training for the elastic net is justified geometrically in such practical covariate shift problems the approach works by obtaining adjusted coefficients for unlabeled prediction which recalibrate the supervised elastic net to compromise i maintaining elastic net predictions on the labeled data with ii shrinking unlabeled predictions to zero our approach is shown to dominate linear supervised alternatives on unlabeled response predictions when the unlabeled feature data are concentrated on a low dimensional manifold away from the labeled data and the true coefficient vector emphasizes directions away from this manifold large variance of the supervised predictions on the unlabeled set is reduced more than the increase in squared bias when the unlabeled responses are expected to be small so an improved compromise within the bias variance tradeoff is the rationale for this performance improvement performance is validated on simulated and real data

structural tests in additive regression we consider the component analysis problem for a regression model with an additive structure the problem is to test whether some of the additive components are of polynomial structure e g linear without specifying the structure of the remaining components a particular case is the problem of selecting the significant covariates the method that we present is based on the wavelet transform using the haar basis which allows for applications under mild conditions on the design and smoothness of the regression function the results demonstrate that each component of the model can be tested with the rate corresponding to the case if all of the remaining components were known the proposed procedure is also computationally straightforward simulation results and a real data example about female labor supply demonstrate the test s good performance

estimating the lyapunov exponent of a chaotic system with nonparametric regression abstract we discuss procedures based on nonparametric regression for estimating the dominant lyapunov exponent 1 from time series data generated by a nonlinear autoregressive system with additive noise for systems with bounded fluctuations 1 0 is the defining feature of chaos thus our procedures can be used to examine time series data for evidence of chaotic dynamics we show that a consistent estimator of the partial derivatives of the autoregression function can be used to obtain a consistent estimator of 1 the rate of convergence we establish is quite slow a better rate of convergence is derived heuristically and supported by simulations simulation results from several implementations one local thin plate splines and three global neural nets radial basis functions and projection pursuit are presented for two deterministic chaotic systems local splines and neural nets yield accurate estimates of the lyapunov exponent however the spline method is sensitive to the choice of the emb

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

optimal intensity control of a queueing system with state dependent capacity limit a general single state queueing system in which the input and output processes are modeled as point processes with stochastic intensities is studied the problem is to control both the input and the output intensities subject to some state dependent capacity limits and the objective is to maximize a discounted value function with reasonable assumptions on the capacity limits it is shown that there exists an optimal control that is of the threshold type characterized by a finite upper barrier the lower barrier being zero the results developed provide theoretical justification for the optimality of the threshold control which is widely applied in practice

coupled kernel embedding for low resolution face image recognition practical video scene and face recognition systems are sometimes confronted with low resolution lr images the faces may be very small even if the video is clear thus it is difficult to directly measure the similarity between the faces and the high resolution hr training samples face recognition based on traditional super resolution sr methods usually have limited performance because the target of sr may not be consistent with that of classification and time consuming sr algorithms are not suitable for real time applications in this paper a new feature extraction method called coupled kernel embedding cke is proposed for lr face recognition without any sr preprocessing in this method the final kernel matrix is constructed by concatenating two individual kernel matrices in the diagonal direction and the semi positively definite properties are preserved for optimization cke addresses the problem of comparing multimodal data that are difficult for conventional methods in practice due to the lack of an efficient similarity measure particularly different kernel types e g linear gaussian polynomial can be integrated into a uniform optimization objective which cannot be achieved by simple linear methods cke solves this problem by minimizing the dissimilarities captured by their kernel gram matrices in the lr and hr spaces in the implementation the nonlinear objective function is minimized by a generalized eigenvalue decomposition experiments on benchmark and real databases show that our cke method indeed improves the recognition performance

multiple kernel learning algorithms in recent years several methods have been proposed to combine multiple kernels instead of using a single one these different kernels may correspond to using different notions of similarity or may be using information coming from multiple sources different representations or different feature subsets in trying to organize and highlight the similarities and differences between them we give a taxonomy of and review several multiple kernel learning algorithms we perform experiments on real data sets for better illustration and comparison of existing algorithms we see that though there may not be large differences in terms of accuracy there is difference between them in complexity as given by the number of stored support vectors the sparsity of the solution as given by the number of used kernels and training time complexity we see that overall using multiple kernels instead of a single one is useful and believe that combining kernels in a nonlinear or data dependent way seems more promising than linear combination in fusing information provided by simple linear kernels whereas linear methods are more reasonable when combining complex gaussian kernels

dimensionality reduction by mixed kernel canonical correlation analysis in this paper we propose a novel method named mixed kernel cca mkcca to achieve easy yet accurate implementation of dimensionality reduction mkcca consists of two major steps first the high dimensional data space is mapped into the reproducing kernel hilbert space rkhs rather than the hilbert space with a mixture of kernels i e a linear combination between a local kernel and a global kernel meanwhile a uniform design for experiments with mixtures is also introduced for model selection second in the new rkhs kernel cca is further improved by performing principal component analysis pca followed by cca for effective dimensionality reduction we prove that mkcca can actually be decomposed into two separate components i e pca and cca which can be used to better remove noises and tackle the issue of trivial learning existing in cca or traditional kernel cca after this the proposed mkcca can be implemented in multiple types of learning such as multi view learning supervised learning semi supervised learning and transfer learning with the reduced data we show its superiority over existing methods in different types of learning by extensive experimental results