totally geodesic foliations on 4 manifolds we give a rather detailed description of the behavior of 2 dimensional totally geodesic foliations on compact riemannian 4 manifolds in particular we obtain a complete characterization in the simply connected case

multiple kernel learning for dimensionality reduction in solving complex visual learning tasks adopting multiple descriptors to more precisely characterize the data has been a feasible way for improving performance the resulting data representations are typically high dimensional and assume diverse forms hence finding a way of transforming them into a unified space of lower dimension generally facilitates the underlying tasks such as object recognition or clustering to this end the proposed approach termed mkl dr generalizes the framework of multiple kernel learning for dimensionality reduction and distinguishes itself with the following three main contributions first our method provides the convenience of using diverse image descriptors to describe useful characteristics of various aspects about the underlying data second it extends a broad set of existing dimensionality reduction techniques to consider multiple kernel learning and consequently improves their effectiveness third by focusing on the techniques pertaining to dimensionality reduction the formulation introduces a new class of applications with the multiple kernel learning framework to address not only the supervised learning problems but also the unsupervised and semi supervised ones

equivariant basic cohomology of riemannian foliations the basic cohomology of a riemannian foliation on a complete manifold with all leaves closed is the cohomology of the leaf space in this paper we introduce various methods to compute the basic cohomology in the presence of both closed and non closed leaves in the simply connected case or more generally for killing foliations we show that the total basic betti number of the union c of the closed leaves is smaller than or equal to the total basic betti number of the foliated manifold and we give sufficient conditions for equality if there is a basic morse bott function with critical set equal to c we can compute the basic cohomology explicitly another case in which the basic cohomology can be determined is if the space of leaf closures is a simple convex polytope our results are based on molino s observation that the existence of non closed leaves yields a distinguished transverse action on the foliated manifold with fixed point set c we introduce equivariant basic cohomology of transverse actions in analogy to equivariant cohomology of lie group actions enabling us to transfer many results from the theory of lie group actions to riemannian foliations the prominent role of the fixed point set in the theory of torus actions explains the relevance of the set c in the basic setting

the geometry of symplectic pairs we study the geometry of manifolds carrying symplectic pairs consisting of two closed 2 forms of constant ranks whose kernel foliations are complementary using a variation of the construction of boothby and wang we build contact symplectic and contact pairs from symplectic pairs

the subspace information criterion for infinite dimensional hypothesis spaces a central problem in learning is selection of an appropriate model this is typically done by estimating the unknown generalization errors of a set of models to be selected from and then choosing the model with minimal generalization error estimate in this article we discuss the problem of model selection and generalization error estimation in the context of kernel regression models e g kernel ridge regression kernel subset regression or gaussian process regression previously a non asymptotic generalization error estimator called the subspace information criterion sic was proposed that could be successfully applied to finite dimensional subspace models sic is an unbiased estimator of the generalization error for the finite sample case under the conditions that the learning target function belongs to a specified reproducing kernel hilbert space rkhs h and the reproducing kernels centered on training sample points span the whole space h these conditions hold only if dim h l where l is the number of training examples therefore sic could be applied only to finite dimensional rkhss in this paper we extend the range of applicability of sic and show that even if the reproducing kernels centered on training sample points do not span the whole space h sic is an unbiased estimator of an essential part of the generalization error our extension allows the use of any rkhss including infinite dimensional ones i e richer function classes commonly used in gaussian processes support vector machines or boosting we further show that when the kernel matrix is invertible sic can be expressed in a much simpler form making its computation highly efficient in computer simulations on ridge parameter selection with real and artificial data sets sic is compared favorably with other standard model selection techniques for instance leave one out cross validation or an empirical bayesian method

lichnerowicz jacobi cohomology in this paper we extend the notion of lichnerowicz poisson cohomology to jacobi manifolds we study the relation of the so called lichnerowicz jacobi cohomology with the basic de rham cohomology and the cohomology of the lie algebra of functions relative to the representation defined by the hamiltonian vector fields a natural pairing with the canonical homology is constructed the relation between the lichnerowicz poisson cohomology of a quantizable poisson manifold and the lichnerowicz jacobi cohomology of the total space of a prequantization bundle is obtained particular cases of cosymplectic contact and locally conformal symplectic manifolds are discussed finally the lichnerowicz jacobi cohomology of a non transitive example is studied

learning the kernel with hyperkernels this paper addresses the problem of choosing a kernel suitable for estimation with a support vector machine hence further automating machine learning this goal is achieved by defining a reproducing kernel hilbert space on the space of kernels itself such a formulation leads to a statistical estimation problem similar to the problem of minimizing a regularized risk functional we state the equivalent representer theorem for the choice of kernels and present a semidefinite programming formulation of the resulting optimization problem several recipes for constructing hyperkernels are provided as well as the details of common machine learning problems experimental results for classification regression and novelty detection on uci data show the feasibility of our approach

survival analysis with time varying regression effects using a tree based approach nonproportional hazards often arise in survival analysis as is evident in the data from the international non hodgkin s lymphoma prognostic factors project a tree based method to handle such survival data is developed for the assessment and estimation of time dependent regression effects under a cox type model the tree method approximates the time varying regression effects as piecewise constants and is designed to estimate change points in the regression parameters a fast algorithm that relies on maximized score statistics is used in recursive segmentation of the time axis following the segmentation a pruning algorithm with optimal properties similar to those of classification and regression trees cart is used to determine a sparse segmentation bootstrap resampling is used in correcting for overoptimism due to split point optimization the piecewise constant model is often more suitable for clinical interpretation of the regression parameters than the more flexible spline models the utility of the algorithm is shown on the lymphoma data where we further develop the published international risk index into a time varying risk index for non hodgkin s lymphoma

active learning applied to rcs computations with nonuniform sampling using different objective functions an active learning framework is introduced to reduce the number of frequencies and angles one must consider for wideband monostatic scattering computations or measurements this method is used to optimally select those frequencies and angles that would be most informative resulting in nonuniform sampling and often a reduced number of points vis agrave vis uniform sampling in this paper we focus on jointly two dimensional optimal sampling in frequency and incident angle thetas for monostatic scattering the method consists of two basic steps one step involves estimation of model parameters using a least square ls algorithm the next step is to optimally choose the next point frequency and thetas for analysis by the computational model or experiment this new point is selected with the goal of reducing uncertainty in the parametric model quantified via the fisher information matrix iterating these two steps a sequence of numerical computations or measurements are performed each at the most informative point for learning the parameters of the associated simpler parametric model this idea is demonstrated here in the context of reducing the number of points frequencies and orientations at which a computational model must be employed and in order to avoid repeatedly gathering samplings at the edge of the input space an alternative objective function is applied which makes the actively selected points closer to a region of interest

explained randomness in proportional hazards models a coefficient of explained randomness analogous to explained variation but for non linear models was presented by kent the construct hinges upon the notion of kullback leibler information gain kent and o quigley developed these ideas obtaining simple multiple and partial coefficients for the situation of proportional hazards regression their approach was based upon the idea of transforming a general proportional hazards model to a specific one of weibull form xu and o quigley developed a more direct approach more in harmony with the semi parametric nature of the proportional hazards model thereby simplifying inference and allowing for instance the use of time dependent covariates a potential drawback to the coefficient of xu and o quigley is its interpretation as explained randomness in the covariate given time an investigator might feel that the interpretation of the kent and o quigley coefficient as a proportion of explained randomness of time given the covariate is preferable one purpose of this note is to indicate that under an independent censoring assumption the two population coefficients coincide thus the simpler inferential setting for xu and o quigley can also be applied to the coefficient of kent and o quigley our second purpose is to point out that a sample based coefficient in common use in the sas statistical package can be interpreted as an estimate of explained randomness when there is no censoring when there is censoring the sas coefficient would not seem satisfactory in that its population counterpart depends on an independent censoring mechanism however there is a quick fix and we argue in favour of its use

learning unrealizable tasks from minimum entropy queries in supervised learning learning from queries rather than from random examples can improve generalization performance significantly we study the performance of query learning for unrealizable tasks where the student cannot learn from the perfectly as a simple model scenario of this kind we consider a linear perceptron student learning a general nonlinear perceptron teacher two kinds of queries for maximum information gain i e minimum entropy are investigated minimum student space entropy msse queries which are appropriate if the teacher space is unknown and minimum teacher space entropy mtse queries which can be used if the teacher space is assumed to be known but a student of a simpler form has deliberately been chosen we find that for msse queries the structure of the student space determines the efficacy of query learning mtse queries on the other hand which we investigate for the extreme case of a binary perceptron teacher lead to a higher generalization error than random examples due to a lack of feedback about the progress of the student in the way queries are selected

on implementing a primal dual interior point method for conic quadratic optimization based on the work of the nesterov and todd on self scaled cones an implementation of a primal dual interior point method for solving large scale sparse conic quadratic optimization problems is presented the main features of the implementation are it is based on a homogeneous and self dual model it handles rotated quadratic cones directly it employs a mehrotra type predictor corrector extension and sparse linear algebra to improve the computational efficiency finally the implementation exploits fixed variables which naturally occurs in many conic quadratic optimization problems this is a novel feature for our implementation computational results are also presented to document that the implementation can solve very large problems robustly and efficiently

near optimal sensor placement for linear inverse problems a classic problem is the estimation of a set of parameters from measurements collected by only a few sensors the number of sensors is often limited by physical or economical constraints and their placement is of fundamental importance to obtain accurate estimates unfortunately the selection of the optimal sensor locations is intrinsically combinatorial and the available approximation algorithms are not guaranteed to generate good solutions in all cases of interest we propose framesense a greedy algorithm for the selection of optimal sensor locations the core cost function of the algorithm is the frame potential a scalar property of matrices that measures the orthogonality of its rows notably framesense is the first algorithm that is near optimal in terms of mean square error meaning that its solution is always guaranteed to be close to the optimal one moreover we show with an extensive set of numerical experiments that framesense achieves state of the art performance while having the lowest computational cost when compared to other greedy methods

information based objective functions for active data selection learning can be made more efficient if we can actively select particularly salient data points within a bayesian learning framework objective functions are discussed that measure the expected informativeness of candidate measurements three alternative specifications of what we want to gain information about lead to three different criteria for data selection all these criteria depend on the assumption that the hypothesis space is correct which may prove to be their main weakness

efficient hyperkernel learning using second order cone programming the kernel function plays a central role in kernel methods most existing methods can only adapt the kernel parameters or the kernel matrix based on empirical data recently ong et al introduced the method of hyperkernels which can be used to learn the kernel function directly in an inductive setting however the associated optimization problem is a semidefinite program sdp which is very computationally expensive even with the recent advances in interior point methods in this paper we show that this learning problem can be equivalently reformulated as a second order cone program socp which can then be solved more efficiently than sdps comparison is also made with the kernel matrix learning method proposed by lanckriet et al experimental results on both classification and regression problems with toy and real world data sets show that our proposed socp formulation has significant speedup over the original sdp formulation moreover it yields better generalization than lanckriet et al s method with a speed that is comparable or sometimes even faster than their quadratically constrained quadratic program qcqp formulation

finiteness of the basic intersection cohomology of a killing foliation we prove that the basic intersection cohomology mathbb h overline p m mathcal f where mathcal f is the singular foliation determined by an isometric action of a lie group g on the compact manifold m is finite dimensional

how to modify a neural network gradually without changing its input output functionality it is generally unknown when distinct neural networks having different synaptic weights and thresholds implement identical input output transformations determining the exact conditions for structurally distinct yet functionally equivalent networks may shed light on the theoretical constraints on how diverse neural circuits might develop and be maintained to serve identical functions such consideration also imposes practical limits on our ability to uniquely infer the structure of underlying neural circuits from stimulus response measurements we introduce a biologically inspired mathematical method for determining when the structure of a neural network can be perturbed gradually while preserving functionality we show that for common three layer networks with convergent and nondegenerate connection weights this is possible only when the hidden unit gains are power functions exponentials or logarithmic functions which are known to approximate the gains seen in some biological neurons for practical applications our numerical simulations with finite and noisy data show that continuous confounding of parameters due to network functional equivalence tends to occur approximately even when the gain function is not one of the aforementioned three types suggesting that our analytical results are applicable to more general situations and may help identify a common source of parameter variability in neural network modeling

multi class discriminant kernel learning via convex programming regularized kernel discriminant analysis rkda performs linear discriminant analysis in the feature space via the kernel trick its performance depends on the selection of kernels in this paper we consider the problem of multiple kernel learning mkl for rkda in which the optimal kernel matrix is obtained as a linear combination of pre specified kernel matrices we show that the kernel learning problem in rkda can be formulated as convex programs first we show that this problem can be formulated as a semidefinite program sdp based on the equivalence relationship between rkda and least square problems in the binary class case we propose a convex quadratically constrained quadratic programming qcqp formulation for kernel learning in rkda a semi infinite linear programming silp formulation is derived to further improve the efficiency we extend these formulations to the multi class case based on a key result established in this paper that is the multi class rkda kernel learning problem can be decomposed into a set of binary class kernel learning problems which are constrained to share a common kernel based on this decomposition property sdp formulations are proposed for the multi class case furthermore it leads naturally to qcqp and silp formulations as the performance of rkda depends on the regularization parameter we show that this parameter can also be optimized in a joint framework with the kernel extensive experiments have been conducted and analyzed and connections to other algorithms are discussed

a stochastic self organizing map for proximity data we derive an efficient algorithm for topographic mapping of proximity data tmp which can be seen as an extension of kohonen s self organizing map to arbitrary distance measures the tmp cost function is derived in a baysian framework of folded markov chains for the description of autoencoders it incorporates the data by a dissimilarity matrix and the topographic neighborhood by a matrix of transition probabilities from the principle of maximum entropy a nonfactorizing gibbs distribution is obtained which is approximated in a mean field fashion this allows for maximum likelihood estimation using an expectation maximization algorithm in analogy to the transition from topographic vector quantization to the self organizing map we suggest an approximation to tmp that is computationally more efficient in order to prevent convergence to local minima an annealing scheme in the temperature parameter is introduced for which the critical temperature of the first phase transition is calculated in terms o

some properties of a cohomology group associated to a totally geodesic foliation we consider an oriented totally geodesic foliation o of dimension p on a compact connected riemannian manifold m there is a homological characte rization of the p l case due to d sullivan 16 the codimension one case has been completely classified by e ghys 9 in collaboration with e ghys we have given a detailed description of the p 2 case for 4 manifolds 5 the geometry of the arbitrary dimension p case is examined in 4 continuing this work we consider in this paper the complex at of tangential forms these are the forms along the leaves of that are invariant in directions perpendicu lar to the leaves the exterior derivative on m induces a differential operator d v in a w

one class classification with gaussian processes detecting instances of unknown categories is an important task for a multitude of problems such as object recognition event detection and defect localization this article investigates the use of gaussian process gp priors for this area of research focusing on the task of one class classification we analyze different measures derived from gp regression and approximate gp classification we also study important theoretical connections to other approaches and discuss their underlying assumptions experiments are performed using a large number of datasets and different image kernel functions our findings show that our approaches can outperform the well known support vector data description approach indicating the high potential of gaussian processes for one class classification furthermore we show the suitability of our methods in the area of attribute prediction defect localization bacteria recognition and background subtraction these applications and experiments highlight the easy applicability of our method as well as its state of the art performance compared to established methods

scale affine invariant interest point detectors in this paper we propose a novel approach for detecting interest points invariant to scale and affine transformations our scale and affine invariant detectors are based on the following recent results 1 interest points extracted with the harris detector can be adapted to affine transformations and give repeatable results geometrically stable 2 the characteristic scale of a local structure is indicated by a local extremum over scale of normalized derivatives the laplacian 3 the affine shape of a point neighborhood is estimated based on the second moment matrix our scale invariant detector computes a multi scale representation for the harris interest point detector and then selects points at which a local measure the laplacian is maximal over scales this provides a set of distinctive points which are invariant to scale rotation and translation as well as robust to illumination changes and limited changes of viewpoint the characteristic scale determines a scale invariant region for each point we extend the scale invariant detector to affine invariance by estimating the affine shape of a point neighborhood an iterative algorithm modifies location scale and neighborhood of each point and converges to affine invariant points this method can deal with significant affine transformations including large scale changes the characteristic scale and the affine shape of neighborhood determine an affine invariant region for each point we present a comparative evaluation of different detectors and show that our approach provides better results than existing methods the performance of our detector is also confirmed by excellent matching resultss the image is described by a set of scale affine invariant descriptors computed on the regions associated with our points

optimizing the kernel in the empirical feature space in this paper we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space an euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved employing a data dependent kernel we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space it is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by cristianini extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data and leads to a substantial sometimes significant improvement in the performance of various data classification algorithms

multi output separable gaussian process towards an efficient fully bayesian paradigm for uncertainty quantification computer codes simulating physical systems usually have responses that consist of a set of distinct outputs e g velocity and pressure that evolve also in space and time and depend on many unknown input parameters e g physical constants initial boundary conditions etc furthermore essential engineering procedures such as uncertainty quantification inverse problems or design are notoriously difficult to carry out mostly due to the limited simulations available the aim of this work is to introduce a fully bayesian approach for treating these problems which accounts for the uncertainty induced by the finite number of observations our model is built on a multi dimensional gaussian process that explicitly treats correlations between distinct output variables as well as space and or time the proper use of a separable covariance function enables us to describe the huge covariance matrix as a kronecker product of smaller matrices leading to efficient algorithms for carrying out inference and predictions the novelty of this work is the recognition that the gaussian process model defines a posterior probability measure on the function space of possible surrogates for the computer code and the derivation of an algorithmic procedure that allows us to sample it efficiently we demonstrate how the scheme can be used in uncertainty quantification tasks in order to obtain error bars for the statistics of interest that account for the finite number of observations

characterizing responses of translation invariant neurons to natural stimuli maximally informative invariant dimensions the human visual system is capable of recognizing complex objects even when their appearances change drastically under various viewing conditions especially in the higher cortical areas the sensory neurons reflect such functional capacity in their selectivity for complex visual features and invariance to certain object transformations such as image translation due to the strong nonlinearities necessary to achieve both the selectivity and invariance characterizing and predicting the response patterns of these neurons represents a formidable computational challenge a related problem is that such neurons are poorly driven by randomized inputs such as white noise and respond strongly only to stimuli with complex high order correlations such as natural stimuli here we describe a novel two step optimization technique that can characterize both the shape selectivity and the range and coarseness of position invariance from neural responses to natural stimuli one step in the optimization is finding the template as the maximally informative dimension given the estimated spatial location where the response could have been triggered within each image the estimates of the locations that triggered the response are updated in the next step under the assumption of a monotonic relationship between the firing rate and stimulus projections on the template at a given position the most likely location is the one that has the largest projection on the estimate of the template the algorithm shows quick convergence during optimization and the estimation results are reliable even in the regime of small signal to noise ratios when we apply the algorithm to responses of complex cells in the primary visual cortex v1 to natural movies we find that responses of the majority of cells were significantly better described by translation invariant models based on one template compared with position specific models with several relevant features

a practical bayesian framework for backpropagation networks a quantitative and practical bayesian framework is described for learning of mappings in feedforward networks the framework makes possible 1 objective comparisons between solutions using alternative network architectures 2 objective stopping rules for network pruning or growing procedures 3 objective choice of magnitude and type of weight decay terms or additive regularizers for penalizing large weights etc 4 a measure of the effective number of well determined parameters in a model 5 quantified estimates of the error bars on network parameters and on network output and 6 objective comparisons with alternative learning and interpolation models such as splines and radial basis functions the bayesian evidence automatically embodies occam s razor penalizing overflexible and overcomplex models the bayesian approach helps detect poor underlying assumptions in learning models for learning models well matched to a problem a good correlation between generalization ability and the bayesian evidence is obtained

sensor placement by maximal projection on minimum eigenspace for linear inverse problems this paper presents two new greedy sensor placement algorithms named minimum nonzero eigenvalue pursuit mnep and maximal projection on minimum eigenspace mpme for linear inverse problems with greater emphasis on the mpme algorithm for performance comparison with existing approaches in both mnep and mpme we select the sensing locations one by one in this way the least number of required sensor nodes can be determined by checking whether the estimation accuracy is satisfied after each sensing location is determined for the mpme algorithm the minimum eigenspace is defined as the eigenspace associated with the minimum eigenvalue of the dual observation matrix for each sensing location the projection of its observation vector onto the minimum eigenspace is shown to be monotonically decreasing w r t the worst case error variance wcev of the estimated parameters we select the sensing location whose observation vector has the maximum projection onto the minimum eigenspace of the current dual observation matrix the proposed mpme is shown to be one of the most computationally efficient algorithms our monte carlo simulations showed that mpme outperforms the convex relaxation method the sparsense method and the framesense method in terms of wcev and the mean square error mse of the estimated parameters especially when the number of available sensor nodes is very limited

quantification of the prentice criteria for surrogate endpoints in a recent discussion on the prentice criteria for surrogate endpoints 1989 and the role of meta analyses in this context alonso et al expand on a number of points among these is a suggestion to base a measure of the degree at which an endpoint meets the prentice criteria on a likelihood reduction factor although new in this context the suggestion has been studied elsewhere and the measure of alonso et al as presented requires some modification if it is to be useful in general situations as it stands it depends upon the censoring mechanism even when independent of the failure mechanism this is however easily fixed the measure suggested by alonso et al can be seen to fit in with a well established theory so that once corrected for the presence of censoring it can be employed with confidence of particular interest is the wide availability of a standard sas program producing a coefficient which can be immediately transformed to provide an estimate of the population quantity behind the alonso et al proposal

building sparse multiple kernel svm classifiers the support vector machines svms have been very successful in many machine learning problems however they can be slow during testing because of the possibly large number of support vectors obtained recently wu 2005 proposed a sparse formulation that restricts the svm to use a small number of expansion vectors in this paper we further extend this idea by integrating with techniques from multiple kernel learning mkl the kernel function in this sparse svm formulation no longer needs to be fixed but can be automatically learned as a linear combination of kernels two formulations of such sparse multiple kernel classifiers are proposed the first one is based on a convex combination of the given base kernels while the second one uses a convex combination of the so called ldquoequivalentrdquo kernels empirically the second formulation is particularly competitive experiments on a large number of toy and real world data sets show that the resultant classifier is compact and accurate and can also be easily trained by simply alternating linear program and standard svm solver

an information based learning approach to dual control dual control aims to concurrently learn and control an unknown system however actively learning the system conflicts directly with any given control objective for it will disturb the system during exploration this paper presents a receding horizon approach to dual control where a multiobjective optimization problem is solved repeatedly and subject to constraints representing system dynamics balancing a standard finite horizon control objective a knowledge gain objective is defined to explicitly quantify the information acquired when learning the system dynamics measures from information theory such as entropy based uncertainty fisher information and relative entropy are studied and used to quantify the knowledge gained as a result of the control actions the resulting iterative framework is applied to markov decision processes and discrete time nonlinear systems thus the broad applicability and usefulness of the presented approach is demonstrated in diverse problem settings the framework is illustrated with multiple numerical examples

nonnegative matrix factorization in polynomial feature space plenty of methods have been proposed in order to discover latent variables features in data sets such approaches include the principal component analysis pca independent component analysis ica factor analysis fa etc to mention only a few a recently investigated approach to decompose a data set with a given dimensionality into a lower dimensional space is the so called nonnegative matrix factorization nmf its only requirement is that both decomposition factors are nonnegative to approximate the original data the minimization of the nmf objective function is performed in the euclidean space where the difference between the original data and the factors can be minimized by employing l 2 norm in this paper we propose a generalization of the nmf algorithm by translating the objective function into a hilbert space also called feature space under nonnegativity constraints with the help of kernel functions we developed an approach that allows high order dependencies between the basis images while keeping the nonnegativity constraints on both basis images and coefficients two practical applications namely facial expression and face recognition show the potential of the proposed approach

proportional hazards models with frailties and random effects we discuss some of the fundamental concepts underlying the development of frailty and random effects models in survival one of these fundamental concepts was the idea of a frailty model where each subject has his or her own disposition to failure their so called frailty additional to any effects we wish to quantify via regression although the concept of individual frailty can be of value when thinking about how data arise or when interpreting parameter estimates in the context of a fitted model we argue that the concept is of limited practical value individual random effects frailties whenever detected can be made to disappear by elementary model transformation in consequence unless we are to take some model form as unassailable beyond challenge and carved in stone and if we are to understand the term frailty as referring to individual random effects then frailty models have no value random effects models on the other hand in which groups of individuals share some common effect can be used to advantage even in this case however if we are prepared to sacrifice some efficiency we can avoid complex modelling by using the considerable power already provided by the stratified proportional hazards model stratified models and random effects models can both be seen to be particular cases of partially proportional hazards models a view that gives further insight the added structure of a random effects model viewed as a stratified proportional hazards model with some added distributional constraints will for group sizes of five or more provide no more than modest efficiency gains even when the additional assumptions are exactly true on the other hand for moderate to large numbers of very small groups of sizes two or three the study of twins being a well known example the efficiency gains of the random effects model can be far from negligible for such applications the case for using random effects models rather than the stratified model is strong this is especially so in view of the good robustness properties of random effects models nonetheless the simpler analysis based upon the stratified model remains valid albeit making a less efficient use of resources

active state estimation for nonlinear systems a neural approximation approach in this paper we consider the problem of actively providing an estimate of the state of a stochastic dynamic system over a possibly long finite time horizon the active estimation problem aep is formulated as a stochastic optimal control one in which the minimization of a suitable uncertainty measure is carried out toward this end the use of the renyi entropy as an information measure is proposed and motivated a neural control scheme based on the application of the extended ritz method erim and on the use of a gaussian sum filter gsf is then presented simulation results show the effectiveness of the proposed approach

lagrangian support vector machines an implicit lagrangian for the dual of a simple reformulation of the standard quadratic program of a linear support vector machine is proposed this leads to the minimization of an unconstrained differentiable convex function in a space of dimensionality equal to the number of classified points this problem is solvable by an extremely simple linearly convergent lagrangian support vector machine lsvm algorithm lsvm requires the inversion at the outset of a single matrix of the order of the much smaller dimensionality of the original input space plus one the full algorithm is given in this paper in 11 lines of matlab code without any special optimization tools such as linear or quadratic programming solvers this lsvm code can be used as is to solve classification problems with millions of points for example 2 million points in 10 dimensional input space were classified by a linear surface in 82 minutes on a pentium iii 500 mhz notebook with 384 megabytes of memory and additional swap space and in 7 minutes on a 250 mhz ultrasparc ii processor with 2 gigabytes of memory other standard classification test problems were also solved nonlinear kernel classification can also be solved by lsvm although it does not scale up to very large problems it can handle any positive semidefinite kernel and is guaranteed to converge a short matlab code is also given for nonlinear kernels and tested on a number of problems

minimax bounds for active learning this paper analyzes the potential advantages and theoretical challenges of active learning algorithms active learning involves sequential sampling procedures that use information gleaned from previous samples in order to focus the sampling and accelerate the learning process relative to passive learning algorithms which are based on nonadaptive usually random samples there are a number of empirical and theoretical results suggesting that in certain situations active learning can be significantly more effective than passive learning however the fact that active learning algorithms are feedback systems makes their theoretical analysis very challenging this paper aims to shed light on achievable limits in active learning using minimax analysis techniques we study the achievable rates of classification error convergence for broad classes of distributions characterized by decision boundary regularity and noise conditions the results clearly indicate the conditions under which one can expect significant gains through active learning furthermore we show that the learning rates derived are tight for boundary fragment classes in d dimensional feature spaces when the feature marginal density is bounded from above and below

applications of second order cone programming in a second order cone program socp a linear function is minimized over the intersection of an affine set and the product of second order quadratic cones socps are nonlinear convex problems that include linear and convex quadratic programs as special cases but are less general than semidefinite programs sdps several efficient primaldual interior point methods for socp have been developed in the last few years after reviewing the basic theory of socps we describe general families of problems that tan be recast as socps these include robust linear programming and robust leastsquares problems problems involving sums or maxima of norms or with convex hyperbolic constraints we discuss a variety of engineering applications such as filter design antenna array weight design truss design and grasping forte optimization in robotics we describe an efficient primaldual interior point method for solving socps which shares many of the features of primaldual interior point methods for linear program

local distance functions a taxonomy new algorithms and an evaluation we present a taxonomy for local distance functions where most existing algorithms can be regarded as approximations of the geodesic distance defined by a metric tensor we categorize existing algorithms by how where and when they estimate the metric tensor we also extend the taxonomy along each axis how we introduce hybrid algorithms that use a combination of techniques to ameliorate overfitting where we present an exact polynomial time algorithm to integrate the metric tensor along the lines between the test and training points under the assumption that the metric tensor is piecewise constant when we propose an interpolation algorithm where the metric tensor is sampled at a number of references points during the offline phase the reference points are then interpolated during the online classification phase we also present a comprehensive evaluation on tasks in face recognition object recognition and digit recognition

active learning and basis selection for kernel based linear models a bayesian perspective we develop an active learning algorithm for kernel based linear regression and classification the proposed greedy algorithm employs a minimum entropy criterion derived using a bayesian interpretation of ridge regression we assume access to a matrix bbrn n for which the i j th element is defined by the kernel function k i j bbr with the observed data i bbrd we seek a model m i yi where yi is a real valued response or integer valued label which we do not have access to a priori to achieve this goal a submatrix il ib bbrn m is sought that corresponds to the intersection of n rows and m columns of indexed by the sets il and ib respectively typically m n and n n we have two objectives i determine the m columns of indexed by the set ib that are the most informative for building a linear model m 1 i ib t yi without any knowledge of yi i 1n and ii using active learning sequentially determine which subset of n elements of yi i 1n should be acquired both stopping values ib m and il n are also to be inferred from the data these steps are taken with the goal of minimizing the uncertainty of the model parameters x as measured by the differential entropy of its posterior distribution the parameter vector x bbrm as well as the model bias bbr is then learned from the resulting problem yil il ibx 1 the remaining n n responses labels not included in yil can be inferred by applying x to the remaining n n rows of ib we show experimental results for several regression and classification problems and compare to other active learning methods

variational bayes for continuous hidden markov models and its application to active learning in this paper we present a variational bayes vb framework for learning continuous hidden markov models chmms and we examine the vb framework within active learning unlike a maximum likelihood or maximum a posteriori training procedure which yield a point estimate of the chmm parameters vb based training yields an estimate of the full posterior of the model parameters this is particularly important for small training sets since it gives a measure of confidence in the accuracy of the learned model this is utilized within the context of active learning for which we acquire labels for those feature vectors for which knowledge of the associated label would be most informative for reducing model parameter uncertainty three active learning algorithms are considered in this paper 1 query by committee qbc with the goal of selecting data for labeling that minimize the classification variance 2 a maximum expected information gain method that seeks to label data with the goal of reducing the entropy of the model parameters and 3 an error reduction based procedure that attempts to minimize classification error over the test data the experimental results are presented for synthetic and measured data we demonstrate that all of these active learning methods can significantly reduce the amount of required labeling compared to random selection of samples for labeling

an interactive approach to solving correspondence problems finding correspondences among objects in different images is a critical problem in computer vision even good correspondence procedures can fail however when faced with deformations occlusions and differences in lighting and zoom levels across images we present a methodology for augmenting correspondence matching algorithms with a means for triaging the focus of attention and effort in assisting the automated matching for guiding the mix of human and automated initiatives we introduce a measure of the expected value of resolving correspondence uncertainties we explore the value of the approach with experiments on benchmark data

new support vector algorithms we propose a new class of support vector algorithms for regression and classification in these algorithms a parameter lets one effectively control the number of support vectors while this can be useful in its own right the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm the accuracy parameter epsilon in the regression case and the regularization constant c in the classification case we describe the algorithms give some theoretical results concerning the meaning and the choice of and report experimental results

second order cone programming second order cone programming socp problems are convex optimization problems in which a linear function is minimized over the intersection of an affine linear manifold with the cartesian product of second order lorentz cones linear programs convex quadratic programs and quadratically constrained convex quadratic programs can all be formulated as socp problems as can many other problems that do not fall into these three categories these latter problems model applications from a broad range of fields from engineering control and finance to robust optimization and combinatorial optimization on the other hand semidefinite programming sdp that is the optimization problem over the intersection of an affine set and the cone of positive semidefinite matrices includes socp as a special case therefore socp falls between linear lp and quadratic qp programming and sdp like lp qp and sdp problems socp problems can be solved in polynomial time by interior point methods the computational effort per iteration required by these methods to solve socp problems is greater than that required to solve lp and qp problems but less than that required to solve sdp s of similar size and structure because the set of feasible solutions for an socp problem is not polyhedral as it is for lp and qp problems it is not readily apparent how to develop a simplex or simplex like method for socp while socp problems can be solved as sdp problems doing so is not advisable both on numerical grounds and computational complexity concerns for instance many of the problems presented in the survey paper of vandenberghe and boyd vb96 as examples of sdps can in fact be formulated as socps and should be solved as such in 2 3 below we give socp formulations for four of these examples the convex quadratically constrained quadratic programming qcqp problem problems involving fractional quadratic functions rutcor rutgers university e mail alizadeh rutcor rutgers edu research supported in part by the u s national science foundation grant ccr 9901991 ieor columbia university e mail gold ieor columbia edu research supported in part by the department of energy grant de fg02 92er25126 national science foundation grants dms 94 14438 cda 97 26385 and dms 01 04282

metric and kernel learning using a linear transformation metric and kernel learning arise in several machine learning applications however most existing metric learning algorithms are limited to learning metrics over low dimensional data while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new data points in this paper we study the connections between metric learning and kernel learning that arise when studying metric learning as a linear transformation learning problem in particular we propose a general optimization framework for learning metrics via linear transformations and analyze in detail a special case of our framework that of minimizing the logdet divergence subject to linear constraints we then propose a general regularized framework for learning a kernel matrix and show it to be equivalent to our metric learning framework our theoretical connections between metric and kernel learning have two main consequences 1 the learned kernel matrix parameterizes a linear transformation kernel function and can be applied inductively to new data points 2 our result yields a constructive method for kernelizing most existing mahalanobis metric learning formulations we demonstrate our learning approach by applying it to large scale real world problems in computer vision text mining and semi supervised kernel dimensionality reduction

machine learning of linear differential equations using gaussian processes this work leverages recent advances in probabilistic machine learning to discover governing equations expressed by parametric linear operators such equations involve but are not limited to ordinary and partial differential integro differential and fractional order operators here gaussian process priors are modified according to the particular form of such operators and are employed to infer parameters of the linear equations from scarce and possibly noisy observations such observations may come from experiments or black box computer simulations as demonstrated in several synthetic examples and a realistic application in functional genomics employ probabilistic machine learning to discover governing equations expressed by parametric linear operators proper placement of gaussian process priors allows one to efficiently infer model parameters via maximum likelihood estimation a general treatment of inverse problems governed by linear operators leading to model discovery from just a handful of noisy measurements

sequential optimal design of neurophysiology experiments adaptively optimizing experiments has the potential to significantly reduce the number of trials needed to build parametric statistical models of neural systems however application of adaptive methods to neurophysiology has been limited by severe computational challenges since most neurons are high dimensional systems optimizing neurophysiology experiments requires computing high dimensional integrations and optimizations in real time here we present a fast algorithm for choosing the most informative stimulus by maximizing the mutual information between the data and the unknown parameters of a generalized linear model glm that we want to fit to the neuron s activity we rely on important log concavity and asymptotic normality properties of the posterior to facilitate the required computations our algorithm requires only low rank matrix manipulations and a two dimensional search to choose the optimal stimulus the average running time of these operations scales quadratically with the dimensionality of the glm making real time adaptive experimental design feasible even for high dimensional stimulus and parameter spaces for example we require roughly 10 milliseconds on a desktop computer to optimize a 100 dimensional stimulus despite using some approximations to make the algorithm efficient our algorithm asymptotically decreases the uncertainty about the model parameters at a rate equal to the maximum rate predicted by an asymptotic analysis simulation results show that picking stimuli by maximizing the mutual information can speed up convergence to the optimal values of the parameters by an order of magnitude compared to using random nonadaptive stimuli finally applying our design procedure to real neurophysiology experiments requires addressing the nonstationarities that we would expect to see in neural responses our algorithm can efficiently handle both fast adaptation due to spike history effects and slow nonsystematic drifts in a neuron s activity

a measure of explained variation for event history data there is no shortage of proposed measures of prognostic value of survival models in the statistical literature they come under different names including explained variation correlation explained randomness and information gain but their goal is common to define something analogous to the coefficient of determination r 2 in linear regression none however have been uniformly accepted none have been extended to general event history data including recurrent events and many cannot incorporate time varying effects or covariates we present here a measure specifically tailored for use with general dynamic event history regression models the measure is applicable and interpretable in discrete or continuous time with tied data or otherwise with time varying time fixed or dynamic covariates with time varying or time constant effects with single or multiple event times with parametric or semiparametric models and under general independent censoring observation for single event survival data with neither censoring nor time dependency it reduces to the concordance index we give expressions for its population value and the variance of the estimator and explore its use in simulations and applications a web link to r software is provided

predictive capability of proportional hazards regression a measure of the predictive capability of a proportional hazards regression is derived the measure is based on the residuals appropriate to proportional hazards regression a population version is presented and can be seen not to depend on the censoring mechanism under the provision that any such censoring be independent or conditionally independent of the failure mechanism given the covariate for the special case of a weibull regression model for which the covariate distribution follows binary uniform normal or exponential laws we derive analytic results these alone give credence to the measure which can be seen to reflect strength of regression effect as quantified by the parameter estimate although on a scale between 0 and 1 independently of the intercept or shape parameter of the particular weibull law and only weakly dependent on the covariate distribution extensions to partial and multiple measures of predictive ability are straightforward an example is provided

support vector echo state machine for chaotic time series prediction a novel chaotic time series prediction method based on support vector machines svms and echo state mechanisms is proposed the basic idea is replacing kernel trick with reservoir trick in dealing with nonlinearity that is performing linear support vector regression svr in the high dimension reservoir state space and the solution benefits from the advantages from structural risk minimization principle and we call it support vector echo state machines svesms svesms belong to a special kind of recurrent neural networks rnns with convex objective function and their solution is global optimal and unique svesms are especially efficient in dealing with real life nonlinear time series and its generalization ability and robustness are obtained by regularization operator and robust loss function the method is tested on the benchmark prediction problem of mackey glass time series and applied to some real life time series such as monthly sunspots time series and runoff time series of the yellow river and the prediction results are promising

near optimal sensor placements in gaussian processes theory efficient algorithms and empirical studies when monitoring spatial phenomena which can often be modeled as gaussian processes gps choosing sensor locations is a fundamental task there are several common strategies to address this task for example geometry or disk models placing sensors at the points of highest entropy variance in the gp model and a d or e optimal design in this paper we tackle the combinatorial optimization problem of maximizing the mutual information between the chosen locations and the locations which are not selected we prove that the problem of finding the configuration that maximizes mutual information is np complete to address this issue we describe a polynomial time approximation that is within 1 1 e of the optimum by exploiting the submodularity of mutual information we also show how submodularity can be used to obtain online bounds and design branch and bound search procedures we then extend our algorithm to exploit lazy evaluations and local structure in the gp yielding significant speedups we also extend our approach to find placements which are robust against node failures and uncertainties in the model these extensions are again associated with rigorous theoretical approximation guarantees exploiting the submodularity of the objective function we demonstrate the advantages of our approach towards optimizing mutual information in a very extensive empirical study on two real world data sets

holomorphic foliations in ruled surfaces we analyse the universal families of holomorphic foliations with singularities in a ruled surface in terms of chern classes we determine the general and the special families we also classify all nonsingular foliations a holomorphic foliation with singularities by curves in a complex manifold m may be defined locally as the integral curves of a nonidentically zero holomorphic vector field and globally by a holomorphic bundle map from a line bundle on m to the tangent bundle of m x l tm uniquely defined up to multiplication by a never vanishing holomorphic function the set of all holomorphic foliations by curves in a compact manifold m has a natural structure of a complex analytic variety usually with infinitely many connected components and in case m is a hodge manifold then each connected component is compact 5 the objective of this paper is to describe these components for the case of a ruled surface s we organise the components according to the chern classes of the defining bundle l and we determine those elements in h 2 s z that appear as chern classes of l using the notion of an ample direction b or of the ample cone in h 2 s z we observe a generic behaviour of the components 2m m for m large enough and a special behaviour for m small the pattern observed resembles the pattern obtained in the theory of special divisors on a riemann surface 7 we also determine all nonsingular foliations in ruled surfaces and give topological and analytic descriptions of them we finish by determining which foliations in ruled surfaces accept a change of complex structure on the leaves keeping the transversal structure fixed 6 the paper is organised as follows in 1 we describe the results of ruled surfaces i e holomorphic cpl bundles over a riemann surface b that we will need in 2 we will obtain restrictions on the chern classes c l e h 2 s z of holomorphic foliations x l ts and describe the nonsingular foliations if the genus g of b is distinct from 1 in 3 we analyse the case g 1 in 4 we describe the components as well as the subvariety of that parametrizes foliations that have an algebraic leaf of fixed fundamental class finally in 5 we apply the variational method of 6 to the ruled surface case received by the editors october 15 1987 1980 mathematics subject classification 1985 revision primary 58f18 14j26 supported by conacyt and university of warwick 1989 american mathematical society 0002 9947 89 1 00 25 per page

gaussian processes for object categorization discriminative methods for visual object category recognition are typically non probabilistic predicting class labels but not directly providing an estimate of uncertainty gaussian processes gps provide a framework for deriving regression techniques with explicit uncertainty models we show here how gaussian processes with covariance functions defined based on a pyramid match kernel pmk can be used for probabilistic object category recognition our probabilistic formulation provides a principled way to learn hyperparameters which we utilize to learn an optimal combination of multiple covariance functions it also offers confidence estimates at test points and naturally allows for an active learning paradigm in which points are optimally selected for interactive labeling we show that with an appropriate combination of kernels a significant boost in classification performance is possible further our experiments indicate the utility of active learning with probabilistic predictive models especially when the amount of training data labels that may be sought for a category is ultimately very small

large scale gaussian process inference with generalized histogram intersection kernels for visual recognition tasks we present new methods for fast gaussian process gp inference in large scale scenarios including exact multi class classification with label regression hyperparameter optimization and uncertainty prediction in contrast to previous approaches we use a full gaussian process model without sparse approximation techniques our methods are based on exploiting generalized histogram intersection kernels and their fast kernel multiplications we empirically validate the suitability of our techniques in a wide range of scenarios with tens of thousands of examples whereas plain gp models are intractable due to both memory consumption and computation time in these settings our results show that exact inference can indeed be done efficiently in consequence we enable every important piece of the gaussian process framework learning inference hyperparameter optimization variance estimation and online learning to be used in realistic scenarios with more than a handful of data

three measures of explained variation for correlated survival data under the proportional hazards mixed effects model measures of explained variation are useful in scientific research as they quantify the amount of variation in an outcome variable of interest that is explained by one or more other variables we develop such measures for correlated survival data under the proportional hazards mixed effects model because different approaches have been studied in the literature outside the classical linear regression model we investigate three measures r 2 rres2 and 2 that quantify three different population coefficients we show that although the three population measures are not the same they reflect similar amounts of variation explained by the predictors among the three measures we show that r 2 which is the simplest to compute is also consistent for the first population measure under the usual asymptotic scenario when the number of clusters tends to infinity the other two measures on the other hand all require that in addition the cluster sizes be large we study the properties of the measures both analytically and through simulation studies we illustrate their different usage on a multi center clinical trial and a recurrent events data set copyright 2016 john wiley sons ltd

sequential design of computer experiments for the assessment of fetus exposure to electromagnetic fields in this article we describe four sequential sampling strategies for estimating the quantile of y f x where x has a known distribution in rd and f is a deterministic unknown expensive to evaluate real valued function these approaches all consist in modeling f as a sample of a well chosen gaussian process and aim at estimating the quantile by using as few evaluations of f as possible the different methodologies are first compared through various numerical experiments then in the framework of the anr jst fetus project we apply our strategies to a real example corresponding to the exposure of a japanese pregnant woman model and her 26 week old fetus to a plane wave finally we compare our methodologies on a simplified geometric model designed for modeling the fetus exposure to plane waves supplementary materials for this article are available online

covariate shift adaptation by importance weighted cross validation a common assumption in supervised learning is that the input points in the training set follow the same probability distribution as the input points that will be given in the future test phase however this assumption is not satisfied for example when the outside of the training region is extrapolated the situation where the training input points and test input points follow different distributions while the conditional distribution of output values given input points is unchanged is called the covariate shift under the covariate shift standard model selection techniques such as cross validation do not work as desired since its unbiasedness is no longer maintained in this paper we propose a new method called importance weighted cross validation iwcv for which we prove its unbiasedness even under the covariate shift the iwcv procedure is the only one that can be applied for unbiased classification under covariate shift whereas alternatives to iwcv exist for regression the usefulness of our proposed method is illustrated by simulations and furthermore demonstrated in the brain computer interface where strong non stationarity effects can be seen between training and test sessions

bayesian computation using design of experiments based interpolation technique in this article a new deterministic approximation method for bayesian computation known as design of experiments based interpolation technique doit is proposed the method works by sampling points from the parameter space using an experimental design and then fitting a kriging model to interpolate the unnormalized posterior the approximated posterior density is a weighted average of normal densities and therefore most of the posterior quantities can be easily computed doit is a general computing technique that is easy to implement and can be applied to many complex bayesian problems moreover it does not suffer from the curse of dimensionality as much as some quadrature methods it can work using fewer posterior evaluations which is a great advantage over the monte carlo and markov chain monte carlo methods especially when dealing with computationally expensive posteriors this article has supplementary material that is available online

the evidence framework applied to classification networks three bayesian ideas are presented for supervised adaptive classifiers first it is argued that the output of a classifier should be obtained by marginalizing over the posterior distribution of the parameters a simple approximation to this integral is proposed and demonstrated this involves a moderation of the most probable classifier s outputs and yields improved performance second it is demonstrated that the bayesian framework for model comparison described for regression models in mackay 1992a b can also be applied to classification problems this framework successfully chooses the magnitude of weight decay terms and ranks solutions found using different numbers of hidden units third an information based data selection criterion is derived and demonstrated within this framework

multi output local gaussian process regression applications to uncertainty quantification we develop an efficient bayesian uncertainty quantification framework using a novel treed gaussian process model the tree is adaptively constructed using information conveyed by the observed data about the length scales of the underlying process on each leaf of the tree we utilize bayesian experimental design techniques in order to learn a multi output gaussian process the constructed surrogate can provide analytical point estimates as well as error bars for the statistics of interest we numerically demonstrate the effectiveness of the suggested framework in identifying discontinuities local features and unimportant dimensions in the solution of stochastic differential equations

adaptive hessian based nonstationary gaussian process response surface method for probability density approximation with application to bayesian solution of large scale inverse problems we develop an adaptive hessian based non stationary gaussian process gp response surface method for approximating a probability density function pdf that exploits its structure particularly the hessian of its negative logarithm of particular interest to us are pdfs that arise from the bayesian solution of large scale inverse problems which imply very expensive to evaluate pdfs the method can be considered as a piecewise adaptive gaussian approximation in which a gaussian tailored to the local hessian of the negative log probability density is constructed for each subregion in high dimensional parameter space the task of efficiently partitioning the parameter space into subregions is done implicitly through hessian informed membership probability functions the gp machinery is then employed to glue all local gaussian approximations into a global analytical response surface that is far cheaper to evaluate than the original expensive probability density the resulting response surface is also equipp

efficient svm training using low rank kernel representations svm training is a convex optimization problem which scales with the training set size rather than the feature space dimension while this is usually considered to be a desired quality in large scale problems it may cause training to be impractical the common techniques to handle this difficulty basically build a solution by solving a sequence of small scale subproblems our current effort is concentrated on the rank of the kernel matrix as a source for further enhancement of the training procedure we first show that for a low rank kernel matrix it is possible to design a better interior point method ipm in terms of storage requirements as well as computational complexity we then suggest an efficient use of a known factorization technique to approximate a given kernel matrix by a low rank matrix which in turn will be used to feed the optimizer finally we derive an upper bound on the change in the objective function value based on the approximation error and the number of active constraints support vectors this bound is general in the sense that it holds regardless of the approximation method

a decomposition theorem for the spectral sequence of lie foliations for a lie g foliation y on a closed manifold m there is an infinitesimal action of g on m up to homotopy along the leaves in general it is not an action but defines an action of the corresponding connected simply connected lie group on the term e1 of the spectral sequence associated to s9 even though e1 in general is infinite dimensional and non hausdorff with the topology induced by the f topology it is proved that this action can be averaged when is compact obtaining a tensor decomposition theorem of e2 it implies duality in the whole term e2 for riemannian foliations on closed oriented manifolds with compact semisimple structural lie algebra

bayesian active learning of neural firing rate maps with transformed gaussian process priors a firing rate map also known as a tuning curve describes the nonlinear relationship between a neuron s spike rate and a low dimensional stimulus e g orientation head direction contrast color here we investigate bayesian active learning methods for estimating firing rate maps in closed loop neurophysiology experiments these methods can accelerate the characterization of such maps through the intelligent adaptive selection of stimuli specifically we explore the manner in which the prior and utility function used in bayesian active learning affect stimulus selection and performance our approach relies on a flexible model that involves a nonlinearly transformed gaussian process gp prior over maps and conditionally poisson spiking we show that infomax learning which selects stimuli to maximize the information gain about the firing rate map exhibits strong dependence on the seemingly innocuous choice of nonlinear transformation function we derive an alternate utility function that selects stimuli to minimize the average posterior variance of the firing rate map and analyze the surprising relationship between prior parameterization stimulus selection and active learning performance in gp poisson models we apply these methods to color tuning measurements of neurons in macaque primary visual cortex

estimating the support of a high dimensional distribution suppose you are given some data set drawn from an underlying probability distribution p and you want to estimate a simple subset s of input space such that the probability that a test point drawn from p lies outside of s equals some a priori specified value between 0 and 1 we propose a method to approach this problem by trying to estimate a function f that is positive on s and negative on the complement the functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data it is regularized by controlling the length of the weight vector in an associated feature space the expansion coefficients are found by solving a quadratic programming problem which we do by carrying out sequential optimization over pairs of input patterns we also provide a theoretical analysis of the statistical performance of our algorithm the algorithm is a natural extension of the support vector algorithm to the case of unlabeled data

a kernel approach for semisupervised metric learning while distance function learning for supervised learning tasks has a long history extending it to learning tasks with weaker supervisory information has only been studied recently in particular some methods have been proposed for semisupervised metric learning based on pairwise similarity or dissimilarity information in this paper we propose a kernel approach for semisupervised metric learning and present in detail two special cases of this kernel approach the metric learning problem is thus formulated as an optimization problem for kernel learning an attractive property of the optimization problem is that it is convex and hence has no local optima while a closed form solution exists for the first special case the second case is solved using an iterative majorization procedure to estimate the optimal solution asymptotically experimental results based on both synthetic and real world data show that this new kernel approach is promising for nonlinear metric learning

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

dethroning the fano factor a flexible model based approach to partitioning neural variability neurons in many brain areas exhibit high trial to trial variability with spike counts that are overdispersed relative to a poisson distribution recent work goris movshon simoncelli 2014 ha

bayesian classification with gaussian processes we consider the problem of assigning an input vector to one of m classes by predicting p c x for c 1 m for a two class problem the probability of class one given x is estimated by spl sigma y x where spl sigma y 1 1 e sup y a gaussian process prior is placed on y x and is combined with the training data to obtain predictions for new x points we provide a bayesian treatment integrating over uncertainty in y and in the parameters that control the gaussian process prior the necessary integration over y is carried out using laplace s approximation the method is generalized to multiclass problems m 2 using the softmax function we demonstrate the effectiveness of the method on a number of datasets

soft margin multiple kernel learning multiple kernel learning mkl has been proposed for kernel methods by learning the optimal kernel from a set of predefined base kernels however the traditional l 1 rm mkl method often achieves worse results than the simplest method using the average of base kernels i e average kernel in some practical applications in order to improve the effectiveness of mkl this paper presents a novel soft margin perspective for mkl specifically we introduce an additional slack variable called kernel slack variable to each quadratic constraint of mkl which corresponds to one support vector machine model using a single base kernel we first show that l 1 rm mkl can be deemed as hard margin mkl and then we propose a novel soft margin framework for mkl three commonly used loss functions including the hinge loss the square hinge loss and the square loss can be readily incorporated into this framework leading to the new soft margin mkl objective functions many existing mkl methods can be shown as special cases under our soft margin framework for example the hinge loss soft margin mkl leads to a new box constraint for kernel combination coefficients using different hyper parameter values for this formulation we can inherently bridge the method using average kernel l 1 rm mkl and the hinge loss soft margin mkl the square hinge loss soft margin mkl unifies the family of elastic net constraint regularizer based approaches and the square loss soft margin mkl incorporates l 2 rm mkl naturally moreover we also develop efficient algorithms for solving both the hinge loss and square hinge loss soft margin mkl comprehensive experimental studies for various mkl algorithms on several benchmark data sets and two real world applications including video action recognition and event recognition demonstrate that our proposed algorithms can efficiently achieve an effective yet sparse solution for mkl

multiple kernel learning algorithms in recent years several methods have been proposed to combine multiple kernels instead of using a single one these different kernels may correspond to using different notions of similarity or may be using information coming from multiple sources different representations or different feature subsets in trying to organize and highlight the similarities and differences between them we give a taxonomy of and review several multiple kernel learning algorithms we perform experiments on real data sets for better illustration and comparison of existing algorithms we see that though there may not be large differences in terms of accuracy there is difference between them in complexity as given by the number of stored support vectors the sparsity of the solution as given by the number of used kernels and training time complexity we see that overall using multiple kernels instead of a single one is useful and believe that combining kernels in a nonlinear or data dependent way seems more promising than linear combination in fusing information provided by simple linear kernels whereas linear methods are more reasonable when combining complex gaussian kernels

efficient sparse generalized multiple kernel learning kernel methods have been successfully applied in various applications to succeed in these applications it is crucial to learn a good kernel representation whose objective is to reveal the data similarity precisely in this paper we address the problem of multiple kernel learning mkl searching for the optimal kernel combination weights through maximizing a generalized performance measure most mkl methods employ the norm simplex constraints on the kernel combination weights which therefore involve a sparse but non smooth solution for the kernel weights despite the success of their efficiency they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance alternatively imposing the norm constraint on the kernel weights will keep all the information in the base kernels this leads to non sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information to tackle these problems we propose a generalized mkl gmkl model by introducing an elastic net type constraint on the kernel weights more specifically it is an mkl model with a constraint on a linear combination of the norm and the squared norm on the kernel weights to seek the optimal kernel combination weights therefore previous mkl problems based on the norm or the norm constraints can be regarded as special cases furthermore our gmkl enjoys the favorable sparsity property on the solution and also facilitates the grouping effect moreover the optimization of our gmkl is a convex optimization problem where a local solution is the global optimal solution we further derive a level method to efficiently solve the optimization problem a series of experiments on both synthetic and real world datasets have been conducted to show the effectiveness and efficiency of our gmkl