efficient optimization for sparse gaussian process regression we propose an efficient optimization algorithm to select a subset of training data as the inducing set for sparse gaussian process regression previous methods either use different objective functions for inducing set and hyperparameter selection or else optimize the inducing set by gradient based continuous optimization the former approaches are harder to interpret and suboptimal whereas the latter cannot be applied to discrete input domains or to kernel functions that are not differentiable with respect to the input the algorithm proposed in this work estimates an inducing set and the hyperparameters using a single objective it can be used to optimize either the marginal likelihood or a variational free energy space and time complexity are linear in training set size and the algorithm can be applied to large regression problems on discrete or continuous domains empirical evaluation shows state of art performance in discrete cases competitive prediction results as well as a favorable trade off between training and test time in continuous cases

structure adaptive approach for dimension reduction we propose a new method of effective dimension reduction for a multiindex model which is based on iterative improvement of the family of average derivative estimates the procedure is computationally straightforward and does not require any prior information about the structure of the underlying model we show that in the case when the effective dimension m of the index space does not exceed 3 this space can be estimated with the rate n 1 2 under rather mild assumptions on the model

consistency of trace norm minimization regularization by the sum of singular values also referred to as the trace norm is a popular technique for estimating low rank rectangular matrices in this paper we extend some of the consistency results of the lasso to provide necessary and sufficient conditions for rank consistency of trace norm minimization with the square loss we also provide an adaptive version that is rank consistent even when the necessary condition for the non adaptive version is not fulfilled

on slater s criterion for the breakup of invariant curves abstract we numerically explore slater s theorem in the context of dynamical systems to study the breakup of invariant curves slater s theorem states that an irrational translation over a circle returns to an arbitrary interval in at most three different recurrence times expressible by the continued fraction expansion of the related irrational number the hypothesis considered in this paper is that slater s theorem can be also verified in the dynamics of invariant curves hence we use slater s theorem to develop a qualitative and quantitative numerical approach to determine the breakup of invariant curves in the phase space of area preserving maps

extended integrability and bi hamiltonian systems the current notion of integrability of hamiltonian systems was fixed by liouville in a famous 1855 paper it describes systems in a 2k dimensional phase space whose trajectories are dense on tori q or wind on toroidal cylinders m q m within liouville s construction the dimension q cannot exceed k and is the main invariant of the system in this paper we generalize liouville integrability so that trajectories can be dense on tori k of arbitrary dimensions q 1 2k 1 2k and an additional invariant v 2 q k v 2 q 2 can be recovered the main theorem classifies all k k 1 2 canonical forms of hamiltonian systems that are integrable in a newly defined broad sense an integrable physical problem having engineering origin is presented the notion of extended compatibility of two poisson structures is introduced the corresponding bi hamiltonian systems are shown to be integrable in the broad sense

barriers to transport and mixing in volume preserving maps with nonzero flux abstract we identify some geometric structures secondary tori that restrict transport and prevent mixing in perturbations of integrable volume preserving systems with nonzero net flux unlike the customary kam tori secondary tori cannot be continued to the tori present in the integrable system but are generated by resonances and have a contractible direction we also note that secondary tori persist under the addition of a net flux which destroys all customary kam tori we introduce a remarkably simple algorithm to analyze the behavior of volume preserving maps and to obtain quantitative properties of the secondary tori we then implement the algorithm and after running it present assertions regarding the distribution of the escape times of the unbounded orbits the abundance of secondary tori the size of the resonant regions and the robustness of the tori under the addition of a mean flux

direct estimation of the index coefficient in a single index model single index modeling is widely applied in for example econometric studies as a compromise between too restrictive parametric models and flexible but hardly estimable purely nonparametric models by such modeling the statistical analysis usually focuses on estimating the index coefficients the average derivative estimator ade of the index vector is based on the fact that the average gradient of a single index function f x t is proportional to the index vector unfortunately a straightforward application of this idea meets the so called curse of dimensionality problem if the dimensionality d of the model is larger than 2 however prior information about the vector can be used for improving the quality of gradient estimation by extending the weighting kernel in a direction of small directional derivative the method proposed in this paper consists of such iterative improvements of the original ade the whole procedure requires at most 2 log n iterations and the resulting estimator is n consistent under relatively mild assumptions on the model independently of the dimensionality d

weisfeiler lehman graph kernels in this article we propose a family of efficient kernels for large graphs with discrete node labels key to our method is a rapid feature extraction scheme based on the weisfeiler lehman test of isomorphism on graphs it maps the original graph to a sequence of graphs whose node attributes capture topological and label information a family of kernels can be defined based on this weisfeiler lehman sequence of graphs including a highly efficient kernel comparing subtree like patterns its runtime scales only linearly in the number of edges of the graphs and the length of the weisfeiler lehman graph sequence in our experimental evaluation our kernels outperform state of the art graph kernels on several graph classification benchmark data sets in terms of accuracy and runtime our kernels open the door to large scale applications of graph kernels in various disciplines such as computational biology and social network analysis

principal hessian directions revisited abstract li has suggested the method of principal hessian directions for estimating plotting directions that capture curvature in regression functions this article revisits li s proposal offering a number of suggestions for improved application

greene s residue criterion for the breakup of invariant tori of volume preserving maps abstract invariant tori play a fundamental role in the dynamics of symplectic and volume preserving maps codimension one tori are particularly important as they form barriers to transport such tori foliate the phase space of integrable volume preserving maps with one action and d angles for the area preserving case greene s residue criterion is often used to predict the destruction of tori from the properties of nearby periodic orbits even though kam theory applies to the three dimensional case the robustness of tori in such systems is still poorly understood we study a three dimensional reversible volume preserving analogue of chirikov s standard map with one action and two angles we investigate the preservation and destruction of tori under perturbation by computing the residue of nearby periodic orbits we find tori with diophantine rotation vectors in the spiral mean cubic algebraic field the residue is used to generate the critical function of the map and find a candidate for the most robust torus

passive scalars three dimensional volume preserving maps and chaos the dynamics of a medium sized particle passive scalar suspended in a general time periodic incompressible fluid flow can be described by three dimensional volume preserving maps in this paper these maps are studied in limiting cases in which some of the variables change very little in each iteration and others change quite a lot the former are called slow variables or actions the latter fast variables or angles the maps are classified by their number of actions for maps with only one action we find strong evidence for the existence of invariant surfaces that survive the nonlinear perturbation in a kam like way on the other hand for the two action case the motion is confined to invariant lines that break for arbitrary small size of the nonlinearity instead we find that adiabatic invariant surfaces emerge and typically intersect the resonance sheet of the fast motion at these intersections surfaces are locally broken and transitions from one to another can occur we call this process which is analogous to arnold diffusion singularity induced diffusion it is characteristic of two action maps in one action maps this diffusion is blocked by kam like surfaces

breakup of invariant tori for the four dimensional semi standard map abstract we compute the domain of existence of two dimensional invariant tori with fixed frequency vectors for a four dimensional complex symplectic map the map is a generalization of the semi standard map studied by greene and percival it has three parameters a 1 and a 2 representing the strength of the kicks in each degree of freedom and the coupling the domain of existence of a torus in a 1 a 2 is shown to be complete and log convex for fixed k a 1 a 2 explicit bounds on the domain for fixed k are obtained numerical results show that quadratic irrationals can be more robust than the cubic irrational the spiral mean

multidimensional continued fractions dynamical renormalization and kam theory the disadvantage of traditional multidimensional continued fraction algorithms is that it is not known whether they provide simultaneous rational approximations for generic vectors following ideas of dani lagarias and kleinbock margulis we describe a simple algorithm based on the dynamics of flows on the homogeneous space sl d mathbb z backslash sl d mathbb r the space of lattices of covolume one that indeed yields best possible approximations to any irrational vector the algorithm is ideally suited for a number of dynamical applications that involve small divisor problems as an example we explicitly construct a renormalization scheme for the linearization of vector fields on tori of arbitrary dimension

converse kam theory and practice we unify extend reinterpret and apply criteria of birkhoff 1 herman 9 mather 2 3 aubry et al 4 5 and newman and percival 6 for the nonexistence of invariant circles for area preserving twist maps the criteria enable one to establish regions of phase space through which no rotational invariant circles pass for families of maps the same can be done for regions of the combined space of phase points and parameters the criteria can be implemented rigorously on a computer and give a practical method of proving quite strong results as an example we present a computer program which proved that the standard map has no rotational invariant circles for any parameter value k 63 64

birkhoff averages and rotational invariant circles for area preserving maps abstract rotational invariant circles of area preserving maps are an important and well studied example of kam tori john greene conjectured that the locally most robust rotational circles have rotation numbers that are noble i e have continued fractions with a tail of ones and that of these circles the most robust has golden mean rotation number the accurate numerical confirmation of these conjectures relies on the map having a time reversal symmetry and such high accuracy has not been obtained in more general maps in this paper we develop a method based on a weighted birkhoff average for identifying chaotic orbits island chains and rotational invariant circles that does not rely on these symmetries we use chirikov s standard map as our test case and also demonstrate that our methods apply to three other well studied cases

a rigorous partial justification of greene s criterion we prove several theorems that lend support to greene s criterion for the existence or not of invariant circles in twist maps in particular we show that some of the implications of the criterion are correct when the aubry mather sets are smooth invariant circles or uniformly hyperbolic we also suggest a simple modification that can work in the case that the aubry mather sets have nonzero lyapunov exponents the latter is based on a closing lemma for sets with nonzero lyapunov exponents which may have several other applications

time reversal symmetry in dynamical systems a survey abstract in this paper we survey the topic of time reversal symmetry in dynamical systems we begin with a brief discussion of the position of time reversal symmetry in physics after defining time reversal symmetry as it applies to dynamical systems we then introduce a major theme of our survey namely the relation of time reversible dynamical sytems to equivariant and hamiltonian dynamical systems we follow with a survey of the state of the art on the theory of reversible dynamical systems including results on symmetric periodic orbits local bifurcation theory homoclinic orbits and renormalization and scaling some areas of physics and mathematics in which reversible dynamical systems arise are discussed in an appendix we provide an extensive bibliography on the topic of time reversal symmetry in dynamical systems

chaotic boundary of a hamiltonial map abstract the half plane maps are a class of complex hamiltonial maps whose invariant curves at most fixed irrational frequencies can be obtained as a convergent taylor series expansion for these maps the boundary between regular ordered motion on invariant curves and irregular chaotic motion is given by the radius of convergence of the series the successive terms of the series oscillate wildly due to the presence of small divisors methods are presented for taming the series based on the conversion of the convergenceexponentc ln c into the integral of a continuous but nondifferentiable lambda function whose graph whows a similarity structure on small scales self similarity properties are illustrated for the chaotic boundary function c where v is the frequency

sparse on line gaussian processes we develop an approach for sparse representations of gaussian process gp models which are bayesian types of kernel machines in order to overcome their limitations for large data sets the method is based on a combination of a bayesian on line algorithm together with a sequential construction of a relevant subsample of the data that fully specifies the prediction of the gp model by using an appealing parameterization and projection techniques in a reproducing kernel hilbert space recursions for the effective parameters and a sparse gaussian approximation of the posterior process are obtained this allows for both a propagation of predictions and bayesian error measures the significance and robustness of our approach are demonstrated on a variety of experiments

a unifying view of sparse approximate gaussian process regression we provide a new unifying view including all existing proper probabilistic sparse approximations for gaussian process regression our approach relies on expressing the effective prior which the methods are using this allows new insights to be gained and highlights the relationship between existing methods it also allows for a clear theoretically justified ranking of the closeness of the known approximations to the corresponding full gps finally we point directly to designs of new better sparse approximations combining the best of the existing strategies within attractive computational constraints

a framework for evaluating approximation methods for gaussian process regression gaussian process gp predictors are an important component of many bayesian approaches to machine learning however even a straightforward implementation of gaussian process regression gpr requires o n2 space and o n3 time for a data set of n examples several approximation methods have been proposed but there is a lack of understanding of the relative merits of the different approximations and in what situations they are most useful we recommend assessing the quality of the predictions obtained as a function of the compute time taken and comparing to standard baselines e g subset of data and fitc we empirically investigate four different approximation algorithms on four different prediction problems and make our code available to encourage future comparisons

on the estimation of quadratic functionals we discuss the difficulties of estimating quadratic functionals based on observations y t from the white noise model y t int t 0 f u du sigma w t quad t in lbrack 0 1 rbrack where w t is a standard wiener process on lbrack 0 1 rbrack the optimal rates of convergence as sigma rightarrow 0 for estimating quadratic functionals under certain geometric constraints are found specifically the optimal rates of estimating int 1 0 lbrack f k x rbrack 2 dx under hyperrectangular constraints sum f x j f leq cj alpha and weighted l p body constraints sum p f sum infty 1 j r x j f p leq c are computed explicitly where x j f is the j th fourier bessel coefficient of the unknown function f we develop lower bounds based on testing two highly composite hypercubes and address their advantages the attainable lower bounds are found by applying the hardest one dimensional approach as well as the hypercube method we demonstrate that for estimating regular quadratic functionals i e the functionals which can be estimated at rate o sigma 2 the difficulties of the estimation are captured by the hardest one dimensional subproblems and for estimating nonregular quadratic functionals i e no o sigma 2 consistent estimator exists the difficulties are captured at certain finite dimensional the dimension goes to infinity as sigma rightarrow 0 hypercube subproblems