least squares temporal difference methods an analysis under general conditions we consider approximate policy evaluation for finite state and action markov de cision processes mdp with the least squares temporal difference lstd algorithm lstd in an exploration enhanced learning context where policy costs are computed from observations of a markov chain different from the one corresponding to the policy under evaluation we establish for the discounted cost criterion that lstd converges almost surely under mild minimal conditions we also analyze other properties of the iterates involved in the algorithm including convergence in mean and boundedness our analysis draws on theories of both finite space markov chains and weak feller markov chains on a topological space our results can be applied to other temporal difference algorithms and mdp models as examples we give a convergence analysis of a td algorithm and extensions to mdp with compact state and action spaces as well as a convergence proof of a new lstd algorithm with state dependent parameters

on computing puiseux series for multiple imaginary characteristic roots of lti systems with commensurate delays obtaining the puiseux series of multiple imaginary characteristic roots mirs is a fundamental issue in the stability analysis of time delay systems however to the best of the authors knowledge this issue has not been fully investigated up to date this note focuses on the puiseux series expansion of mirs of linear time invariant systems including commensurate delays for an mir of any multiplicity we propose an algorithm for defining the structure of the puiseux series as well as the explicit computation of the corresponding coefficients by using the proposed method we can find all the puiseux series corresponding to all the root loci

vertices from replica in a random matrix theory kontsevich s work on airy matrix integrals has led to explicit results for the intersection numbers of the moduli space of curves in a subsequent work okounkov rederived these results from the edge behavior of a gaussian matrix integral in our work we consider the correlation functions of vertices in a gaussian random matrix theory with an external matrix source we deal with operator products of the form in a expansion for large values of the powers ki in an appropriate scaling limit relating large k s to large n universal scaling functions are derived furthermore we show that the replica method applied to characteristic polynomials of the random matrices together with a duality exchanging n and the number of points provides a new way to recover kontsevich s results on these intersection numbers

invariance properties for a class of quasipolynomials when a time delay system involves multiple imaginary roots mirs the stability analysis will become much more complicated than that in the case with only simple imaginary roots sirs an mir may exhibit different splitting behaviors and to the best of the authors knowledge their properties have not been fully investigated in this paper we focus on characterizing the invariance properties for mirs with any multiplicity furthermore the proposed methodology makes it possible to also cover some degenerate cases already encountered and discussed in the literature in addition we propose an easily implemented frequency sweeping method making it possible to derive the asymptotic behavior without invoking the puiseux series

on actor critic algorithms in this article we propose and analyze a class of actor critic algorithms these are two time scale algorithms in which the critic uses temporal difference learning with a linearly parameterized approximation architecture and the actor is updated in an approximate gradient direction based on information provided by the critic we show that the features for the critic should ideally span a subspace prescribed by the choice of parameterization of the actor we study actor critic algorithms for markov decision processes with polish state and action spaces we state and prove two results regarding their convergence

a counterexample to temporal differences learning sutton s td n method aims to provide a representation of the cost function in an absorbing markov chain with transition costs a simple example is given where the representation obtained depends on a for x 1 the representation is optimal with respect to a least squares error criterion but as x decreases toward 0 the representation becomes progressively worse and in some cases very poor the example suggests a need to understand better the circumstances under which td 0 and q learning obtain satisfactory neural network based compact representations of the cost function a variation of td 0 is also given which performs better on the example

optimal stopping of markov processes hilbert space theory approximation algorithms and an application to pricing high dimensional financial derivatives the authors develop a theory characterizing optimal stopping times for discrete time ergodic markov processes with discounted rewards the theory differs from prior work by its view of per stage and terminal reward functions as elements of a certain hilbert space in addition to a streamlined analysis establishing existence and uniqueness of a solution to bellman s equation this approach provides an elegant framework for the study of approximate solutions in particular the authors propose a stochastic approximation algorithm that tunes weights of a linear combination of basis functions in order to approximate a value function they prove that this algorithm converges almost surely and that the limit of convergence has some desirable properties the utility of the approximation method is illustrated via a computational case study involving the pricing of a path dependent financial derivative security that gives rise to an optimal stopping problem with a 100 dimensional state space

characteristic polynomials of random matrices at edge singularities we have discussed earlier the correlation functions of the random variables mathrm det ensuremath lambda ensuremath x in which x is a random matrix in particular the moments of the distribution of these random variables are universal functions when measured in the appropriate units of the level spacing when the ensuremath lambda mathrm s instead of belonging to the bulk of the spectrum approach the edge a crossover takes place to an airy or to a bessel problem and we consider here these modified classes of universality furthermore when an external matrix source is added to the probability distribution of x various different phenomenona may occur and one can tune the spectrum of this source matrix to other critical points again there are remarkably simple formulas for arbitrary source matrices which allow us to compute the moments of the characteristic polynomials in these cases as well

finite sample analysis of least squares policy iteration in this paper we report a performance bound for the widely used least squares policy iteration lspi algorithm we first consider the problem of policy evaluation in reinforcement learning that is learning the value function of a fixed policy using the least squares temporal difference lstd learning method and report finite sample analysis for this algorithm to do so we first derive a bound on the performance of the lstd solution evaluated at the states generated by the markov chain and used by the algorithm to learn an estimate of the value function this result is general in the sense that no assumption is made on the existence of a stationary distribution for the markov chain we then derive generalization bounds in the case when the markov chain possesses a stationary distribution and is b mixing finally we analyze how the error at each policy evaluation step is propagated through the iterations of a policy iteration method and derive a performance bound for the lspi algorithm

on an airy matrix model with a logarithmic potential the kontsevich penner model an airy matrix model with a logarithmic potential may be derived from a simple gaussian two matrix model through a duality in this dual version the fourier transforms of the n point correlation functions can be computed in the closed form using virasoro constraints we find that in addition to the parameters tn which appear in the korteweg de vries hierarchies one needs to introduce half integer indices tn 2 the free energy as a function of those parameters may be obtained from these virasoro constraints the large n limit follows from the solution to an integral equation this leads to explicit computations for a number of topological invariants

riemannian symmetric superspaces and their origin in random matrix theory gaussian random matrix ensembles defined over the tangent spaces of the large families of cartan s symmetric spaces are considered such ensembles play a central role in mesoscopic physics as they describe the universal ergodic limit of disordered and chaotic single particle systems the generating function for the spectral correlations of each ensemble is reduced to an integral over a riemannian symmetric superspace in the limit of large matrix dimension such a space is defined as a pair g h m r where g h is a complex analytic graded manifold homogeneous with respect to the action of a complex lie supergroup g and m r is a maximal riemannian submanifold of the support of g h

averages of characteristic polynomials in random matrix theory we compute averages of products and ratios of characteristic polynomials associated with orthogonal unitary and symplectic ensembles of random matrix theory the pfaffian determinantal formulae for these averages are obtained and the bulk scaling asymptotic limits are found for ensembles with gaussian weights classical results for the correlation functions of the random matrix ensembles and their bulk scaling limits are deduced from these formulae by a simple computation we employ a discrete approximation method the problem is solved for discrete analogues of random matrix ensembles originating from representation theory and then a limit transition is performed exact pfaffian determinantal formulae for the discrete averages are proven using standard tools of linear algebra no application of orthogonal or skew orthogonal polynomials is needed 2005 wiley periodicals inc

reversals in stability of linear time delay systems a finer characterization abstract in most of the numerical examples of time delay systems proposed in the literature the number of unstable characteristic roots remains positive before and after a multiple critical imaginary root cir appears as the delay seen as a parameter increases this fact may lead to some misunderstandings i a multiple cir may at most affect the instability degree ii it cannot cause any stability reversals stability transitions from instability to stability as far as we know whether the appearance of a multiple cir can induce stability is still unclear in fact when a cir generates a stability reversal has not been specifically investigated in this paper we provide a finer analysis of stability reversals and some new insights into the classification the link between the multiplicity of a cir and the asymptotic behavior with the stabilizing effect based on these results we present an example illustrating that a multiple cir s asymptotic behavior is able to cause a stability reversal to the best of the authors knowledge such an example is a novelty in the literature on time delay systems

recursive construction for a class of radial functions ii superspace we extend the recursion formula for matrix bessel functions which we obtained previously to superspace it is sufficient to do this for the unitary orthosymplectic supergroup by direct computations we show that fairly explicit results can be obtained at least up to dimension 8 8 for the supermatrices since we introduce a new technique we discuss various of its aspects in some detail

derivation of determinantal structures for random matrix ensembles in a new way there are several methods to treat ensembles of random matrices in symmetric spaces circular matrices chiral matrices and others orthogonal polynomials and the supersymmetry method are particular powerful techniques here we present a new approach to calculate averages over ratios of characteristic polynomials at first sight paradoxically one can call our approach supersymmetry without supersymmetry because we use structures from supersymmetry without actually mapping onto superspaces we address two kinds of integrals which cover a wide range of applications for random matrix ensembles for probability densities factorizing in the eigenvalues we find determinantal structures in a unifying way as a new application we derive an expression for the k point correlation function of an arbitrary rotation invariant probability density over the hermitian matrices in the presence of an external field

universality of correlations of levels with discrete statistics we study the statistics of a system of n random levels with integer values in the presence of a logarithmic repulsive potential of dyson type this problem arises in sums over representations young tableaux of gl n in various matrix problems and in the study of statistics of partitions for the permutation group the model is generalized to include an external source and its correlators are found in closed form for any n we reproduce the density of levels in the large n and double scaling limits and the universal correlation functions in dyson s short distance scaling limit we also study the statistics of small levels

an exact method for the stability analysis of time delayed linear time invariant lti systems a general class of linear time invariant systems with time delay is studied recently they attracted considerable interest in the systems and control community the complexity arises due to the exponential type transcendental terms in their characteristic equation the transcendentality brings infinitely many characteristic roots which are cumbersome to elaborate as evident from the literature a number of methodologies have been suggested with limited ability to assess the stability in the parametric domain of time delay this study offers an exact structured and robust methodology to bring a closure to the question at hand ultimately we present a unique explicit analytical expression in terms of the system parameters which not only reveals the stability regions pockets in the domain of time delay but it also declares the number of unstable characteristic roots at any given pocket the method starts with the determination of all possible purely imaginary resonant characteristic roots for any positive time delay to achieve this a simplifying substitution is used for the transcendental terms in the characteristic equation it is proven that the number of such resonant roots for a given dynamics is finite each one of these roots is created by infinitely many time delays which are periodically distributed an interesting property is also claimed next that the root crossing directions at these locations are invariant with respect to the delay and dependent only on the crossing frequency these two unique findings facilitate a simple and practical stability method which is the highlight of the work

arbitrary rotation invariant random matrix ensembles and supersymmetry orthogonal and unitary symplectic case recently the supersymmetry method was extended from gaussian ensembles to arbitrary unitarily invariant matrix ensembles by generalizing the hubbard stratonovich transformation here we complete this extension by including arbitrary orthogonally and unitary symplectically invariant matrix ensembles the results are equivalent to but the approach is different from the superbosonization formula we express our results in a unifying way we also give explicit expressions for all one point functions and discuss features of the higher order correlations

new correlation functions for random matrices and integrals over supergroups the averages of ratios of characteristic polynomials det x of n n random matrices x are investigated in the large n limit for the gue goe and gse ensembles the density of states and the two point correlation function are derived from these ratios the method relies on an extension of the harish chandra itzykson zuber integrals to the goe ensemble and to supergroups which are explicitly computed through heat kernel differential equations an external matrix source linearly coupled to the random matrices may also be added to the gaussian distribution and allows for a discussion of the universality of the goe results in the large n limit

frequency sweeping tests for stability independent of delay in this paper we study the stability properties of linear time invariant delay systems given in a state space form we consider specifically the notion of asymptotic stability independent of delay systems with both commensurate and noncommensurate delays are investigated we present for each class of systems a necessary and sufficient condition in terms of structured singular values and further we demonstrate how these conditions may be extended to study stability independent of delay for uncertain systems our results consist of several frequency sweeping tests that can be systematically implemented and that should complement the previous work

projected equation methods for approximate solution of large linear systems we consider linear systems of equations and solution approximations derived by projection on a low dimensional subspace we propose stochastic iterative algorithms based on simulation which converge to the approximate solution and are suitable for very large dimensional problems the algorithms are extensions of recent approximate dynamic programming methods known as temporal difference methods which solve a projected form of bellman s equation by using simulation based approximations to this equation or by using a projected value iteration method

a frequency sweeping framework for stability analysis of time delay systems for time delay systems the asymptotic behavior analysis of the critical imaginary roots w r t the infinitely many critical delays is an open problem in order to find a general solution we will exploit the link between the asymptotic behavior of critical imaginary roots and the asymptotic behavior of frequency sweeping curves from a new analytic curve perspective as a consequence we will establish a frequency sweeping framework with three main results 1 a finer regularity singularity classification for time delay systems will be obtained 2 the general invariance property will be proved and hence the asymptotic behavior of the critical imaginary roots w r t the infinitely many critical delays can be adequately studied 3 the complete stability problem can be fully solved moreover the frequency sweeping framework is extended to cover a broader class of time delay systems finally the geometric insights of frequency sweeping curves are investigated consequently some deeper properties on the asymptotic behavior of time delay systems and the link to frequency sweeping curves are found

on the averages of characteristic polynomials from classical groups we provide an elementary and self contained derivation of formulae for averages of products and ratios of characteristic polynomials of random matrices from classical groups using classical results due to weyl and littlewood

an eigenvalue perturbation approach to stability analysis part ii when will zeros of time delay systems cross imaginary axis this paper presents an application of the eigenvalue series developed in part i j chen et al siam j control optim 48 2010 pp 5564 5582 to the study of linear time invariant delay systems focusing on the asymptotic behavior of critical characteristic zeros on the imaginary axis we consider systems given in state space form and as quasi polynomials and we develop an eigenvalue perturbation analysis approach which appears to be both conceptually appealing and computationally efficient our results reveal that the zero asymptotic behavior of time delay systems can in general be characterized by solving a simple eigenvalue problem and additionally when described by a quasi polynomial by computing the derivatives of the quasipolynomial

convergence results for some temporal difference methods based on least squares we consider finite state markov decision processes and prove convergence and rate of convergence results for certain least squares policy evaluation algorithms of the type known as lspe lambda these are temporal difference methods for constructing a linear function approximation of the cost function of a stationary policy within the context of infinite horizon discounted and average cost dynamic programming we introduce an average cost method patterned after the known discounted cost method and we prove its convergence for a range of constant stepsize choices we also show that the convergence rate of both the discounted and the average cost methods is optimal within the class of temporal difference methods analysis and experiment indicate that our methods are substantially and often dramatically faster than td lambda as well as more reliable

integration of grassmann variables over invariant functions on flat superspaces we study integration over functions on superspaces these functions are invariant under a transformation which maps the whole superspace onto the part of the superspace which only comprises purely commuting variables we get a compact expression for the differential operator with respect to the commuting variables which results from berezin integration over all grassmann variables also we derive cauchy like integral theorems for invariant functions on supervectors and symmetric supermatrices this extends theorems partly derived by other authors as a physical application we calculate the generating function of the one point correlation function in random matrix theory furthermore we give another derivation of supermatrix bessel functions for u k1 k2

tracking the algebraic multiplicity of crossing imaginary roots for generic quasipolynomials a vandermonde based approach a standard approach in analyzing dynamical systems consists in identifying and understanding the eigenvalues bifurcations when crossing the imaginary axis efficient methods for crossing imaginary roots identification exist however to the best of the author s knowledge the multiplicity of such roots was not deeply investigated in recent papers by the authors 1 2 it is emphasized that the multiplicity of the zero spectral value can exceed the number of the coupled scalar delay differential equations and a constructive approach vandermonde based allowing to an adaptive bound for such a multiplicity is provided namely it is shown that the zero spectral value multiplicity depends on the system structure number of delays and number of non zero coefficients of the associated quasipolynomial rather than the degree of the associated quasipolynomial 3 this technical note extends the constructive approach in investigating the multiplicity of crossing imaginary roots j omega where omega neq 0 and establishes a link with a new class of functional confluent vandermonde matrices a symbolic algorithm for computing the lu factorization for such matrices is provided as a byproduct of the proposed approach a bound sharper than the polya szego generic bound arising from the principle argument is established

comments on stability of linear time invariant fractional delay systems of retarded type in the space of delay parameters automatica 49 2013 1287 1294 this correspondence points out that some results reported in a recent paper mesbahi and haeri 2013 are not correct mainly due to an insufficient consideration of the degenerate case three counterexamples to these results are presented

characteristic polynomials of random matrices number theorists have studied extensively the connections between the distribution of zeros of the riemann function and of some generalizations with the statistics of the eigenvalues of large random matrices it is interesting to compare the average moments of these functions in an interval to their counterpart in random matrices which are the expectation values of the characteristic polynomials of the matrix it turns out that these expectation values are quite interesting for instance the moments of order 2k scale for unitary invariant ensembles as the density of eigenvalues raised to the power k 2 the prefactor turns out to be a universal number i e it is independent of the specific probability distribution an equivalent behaviour and prefactor had been found as a conjecture within number theory the moments of the characteristic determinants of random matrices are computed here as limits at coinciding points of multi point correlators of determinants these correlators are in fact universal in dyson s scaling limit in which the difference between the points goes to zero the size of the matrix goes to infinity and their product remains finite

colored stability crossing sets for siso delay systems this paper considers the stability of a single input single output siso system with an input output i o delay called siso delay system hereafter we first derive an algebraic criterion to solve the complete stability w r t the delay tau next the stability analysis is extended to the case with additional free plant and or controller parameters denoted by a vector x by adopting a mathematical tool for polynomial algebra the discrimination system the complete classification of the critical imaginary roots cirs in the whole x tau parameter space may be obtained furthermore with the above results we present the stability crossing sets scss in a new form called the colored scss for each point on the scs the asymptotic behavior information of the corresponding cir can be coded by a certain color red green or yellow hence the stability in the x tau space can be analyzed easily and intuitively moreover the geometry of the scss can be investigated from a new perspective which will help not only to better understand the stability of the siso delay system but also to considerably simplify the procedure for generating the colored scss

inverted pendulum stabilization characterization of codimension three triple zero bifurcation via multiple delayed proportional gains the paper considers the problem of stabilization of systems possessing a multiple zero eigenvalue at the origin the controller that we propose uses multiple delayed measurements instead of derivative terms doing so we increase the performances of the closed loop in presence of system uncertainties and or noisy measurements the problem formulation and the analysis is presented through a classical engineering problem which is the stabilization of an inverted pendulum on a cart moving horizontally on one hand we perform a nonlinear analysis of the center dynamics described by a three dimensional system of ordinary differential equations with a codimension three triple zero bifurcation on the other hand we present the complementary stability analysis of the corresponding linear time invariant system with two delays describing the behavior around the equilibrium the aim of this analysis is to characterize the possible local bifurcations finally the proposed control scheme is numerically illustrated and discussed

error bounds for approximations from projected linear equations we consider linear fixed point equations and their approximations by projection on a low dimensional subspace we derive new bounds on the approximation error of the solution which are expressed in terms of low dimensional matrices and can be computed by simulation when the fixed point mapping is a contraction as is typically the case in markov decision processes mdp one of our bounds is always sharper than the standard contraction based bounds and another one is often sharper the former bound is also tight in a worst case sense our bounds also apply to the noncontraction case including policy evaluation in mdp with nonstandard projections that enhance exploration there are no error bounds currently available for this case to our knowledge

stability of linear time invariant fractional delay systems of retarded type in the space of delay parameters this paper presents a new method for assessing the bounded input bounded output stability of a class of fractional delay systems with commensurate orders and multiple commensurate delays of retarded type in the proposed method first by mapping the principal sheet of the riemann surface and a pseudo delay transformation an auxiliary polynomial is generated then this auxiliary polynomial is employed to find roots of the characteristic equation on the imaginary axis the properties of the root path close to these roots are used to identify intervals of delay values in which the system is stable the obtained results are illustrated via some numerical examples

recursive construction for a class of radial functions i ordinary space a class of spherical functions is studied which can be viewed as the matrix generalization of bessel functions we derive a recursive structure for these functions we show that they are only special cases of more general radial functions which also have a properly generalized recursive structure some explicit results are worked out for the first time we identify a subclass of such radial functions which consist of a finite number of terms only

delay dependent energy to peak stability of 2 d time delay roesser systems with multiplicative stochastic noises this paper is concerned with the problem of energy to peak stochastic stability epss of two dimensional 2 d roesser systems in the presence of state time varying delays and multiplicative noises first a scheme that ensures a 2 d stochastic time delay system is stochastically stable with an attenuation performance is proposed the scheme presented in this paper can be regarded as an extension of the lyapunov krasovskii functional method for 2 d stochastic time delay systems focusing on the epss problem the proposed scheme is then utilized to derive delay dependent epss conditions in terms of tractable linear matrix inequalities a numerical example is given to illustrate the effectiveness of the derived stability conditions

characteristic polynomials of real symmetric random matrices the correlation functions of the random variables det x in which x is an hermitian n n random matrix are known to exhibit universal local statistics in the large n limit we study here the correlation of those same random variables for real symmetric matrices goe the derivation relies on an exact dual representation of the problem the k point functions are expressed in terms of finite integrals over quaternionic k k matrices however the control of the dyson limit in which the distance of the various parameters s is of the order of the mean spacing requires an integration over the symplectic group it is shown that a generalization of the itzykson zuber method holds for this problem but contrary to the unitary case the semi classical result requires a finite number of corrections to be exact we have also considered the problem of an external matrix source coupled to the random matrix and obtain explicit integral formulae which are useful for the analysis of the large n limit

stability with respect to the delay on a paper of k l cooke and p van den driessche a proposal for the correction of the main result of the title s paper is made some accompanying comments on other points are given so a simpler central result is derived an explicit representation of the set of stable delays is achieved

brief paper average cost temporal difference learning we propose a variant of temporal difference learning that approximates average and differential costs of an irreducible aperiodic markov chain approximations are comprised of linear combinations of fixed basis functions whose weights are incrementally updated during a single endless trajectory of the markov chain we present a proof of convergence with probability 1 and a characterization of the limit of convergence we also provide a bound on the resulting approximation error that exhibits an interesting dependence on the mixing time of the markov chain the results parallel previous work by the authors involving approximations of discounted cost to go

technical communique invariance properties in the root sensitivity of time delay systems with double imaginary roots if i w ir is an eigenvalue of a time delay system for the delay t 0 then i w is also an eigenvalue for the delays t k t 0 k2 p w for any k z we investigate the sensitivity periodicity and invariance properties of the root i w for the case that i w is a double eigenvalue for some t k it turns out that under natural conditions the condition that the root exhibits the completely regular splitting property if the delay is perturbed the presence of a double imaginary root i w for some delay t 0 implies that i w is a simple root for the other delays t k k 0 moreover we show how to characterize the root locus around i w the entire local root locus picture can be completely determined from the square root splitting of the double root we separate the general picture into two cases depending on the sign of a single scalar constant the imaginary part of the first coefficient in the square root expansion of the double eigenvalue

weak convergence properties of constrained emphatic temporal difference learning with constant and slowly diminishing stepsize we consider the emphatic temporal difference td algorithm etd for learning the value functions of stationary policies in a discounted finite state and action markov decision process the etd algorithm was recently proposed by sutton mahmood and white 2016 to solve a long standing divergence problem of the standard td algorithm when it is applied to off policy training where data from an exploratory policy are used to evaluate other policies of interest the almost sure convergence of etd has been proved in our recent work under general off policy training conditions but for a narrow range of diminishing stepsize in this paper we present convergence results for constrained versions of etd with constant stepsize and with diminishing stepsize from a broad range our results characterize the asymptotic behavior of the trajectory of iterates produced by those algorithms and are derived by combining key properties of etd with powerful convergence theorems from the weak convergence methods in stochastic approximation theory for the case of constant stepsize in addition to analyzing the behavior of the algorithms in the limit as the stepsize parameter approaches zero we also analyze their behavior for a fixed stepsize and bound the deviations of their averaged iterates from the desired solution these results are obtained by exploiting the weak feller property of the markov chains associated with the algorithms and by using ergodic theorems for weak feller markov chains in conjunction with the convergence results we get from the weak convergence methods besides etd our analysis also applies to the off policy td algorithm when the divergence issue is avoided by setting sufficiently large it yields for that case new results on the asymptotic convergence properties of constrained off policy td with constant or slowly diminishing stepsize

random matrix singularities and open close intersection numbers the s point correlation function of a gaussian hermitian random matrix model with an external source tuned to generate a multi critical singularity provides the intersection numbers of the moduli space for the p th spin curves through a duality for one marked point the intersection numbers are expressed to all orders in the genus by bessel functions the matrix models for the lie algebras of and provide the intersection numbers of non orientable surfaces the kontsevich penner model and the higher p th airy matrix model with a logarithmic potential are investigated for the open intersection numbers which describe the topological invariants of non orientable surfaces with boundaries string equations for the open closed riemann surface are derived from the structure of the s point correlation functions the gromov witten invariants of the model are evaluated for one marked point as an application of the present method

an analysis of temporal difference learning with function approximation we discuss the temporal difference learning algorithm as applied to approximating the cost to go function of an infinite horizon discounted markov chain the algorithm we analyze updates parameters of a linear function approximator online during a single endless trajectory of an irreducible aperiodic markov chain with a finite or infinite state space we present a proof of convergence with probability one a characterization of the limit of convergence and a bound on the resulting approximation error furthermore our analysis is based on a new line of reasoning that provides new intuition about the dynamics of temporal difference learning in addition to proving new and stronger positive results than those previously available we identify the significance of online updating and potential hazards associated with the use of nonlinear function approximators first we prove that divergence may occur when updates are not based on trajectories of the markov chain this fact reconciles positive and negative results that have been discussed in the literature regarding the soundness of temporal difference learning second we present an example illustrating the possibility of divergence when temporal difference learning is used in the presence of a nonlinear function approximator

level spacing of random matrices in an external source in an earlier work we had considered a gaussian ensemble of random matrices in the presence of a given external matrix source the measure is no longer unitary invariant and the usual techniques based on orthogonal polynomials or on the coulomb gas representation are not available nevertheless the n point correlation functions are still given in terms of the determinant of a kernel known through an explicit integral representation this kernel is no longer symmetric though and is not readily accessible to standard methods in particular finding the level spacing probability is always a delicate problem in fredholm theory and we have to reconsider the problem within our model we find a new class of universality for the level spacing distribution when the spectrum of the source is ajusted to produce a vanishing gap in the density of the state the problem is solved through coupled non linear differential equations which turn out to form a hamiltonian system as a result we find that the level spacing probability p s behaves like exp c s 8 over 3 for large spacing s this is consistent with the asymptotic behavior exp c s 2 beta 2 whenever the density of state behaves near the edge as rho lambda sim lambda beta

logarithmic moments of characteristic polynomials of random matrices abstract in a recent article we have discussed the connections between averages of powers of riemann s function on the critical line and averages of characteristic polynomials of random matrices the result for random matrices was shown to be universal i e independent of the specific probability distribution and the results were derived for arbitrary moments this allows one to extend the previous results to logarithmic moments for which we derive the explicit universal expressions in random matrix theory we then compare these results to various results and conjectures for functions and the correspondence is again striking

spectral form factor in a random matrix theory in the theory of disordered systems the spectral form factor s ensuremath tau the fourier transform of the two level correlation function with respect to the difference of energies is linear for ensuremath tau mathrm ensuremath tau mathrm c and constant for ensuremath tau g mathrm ensuremath tau mathrm c near zero and near mathrm ensuremath tau mathrm c it exhibits oscillations which have been discussed in several recent papers in problems of mesoscopic fluctuations and quantum chaos a comparison is often made with a random matrix theory it turns out that even in the simplest gaussian unitary ensemble these oscillations have not yet been studied there for random matrices the two level correlation function ensuremath rho ensuremath lambda 1 ensuremath lambda 2 exhibits several well known universal properties in the large n limit its fourier transform is linear as a consequence of the short distance universality of ensuremath rho ensuremath lambda 1 ensuremath lambda 2 however the crossover near zero and mathrm ensuremath tau mathrm c requires one to study these correlations for finite n for this purpose we use an exact contour integral representation of the two level correlation function which allows us to characterize these crossover oscillatory properties this representation is then extended to the case in which the hamiltonian is the sum of a deterministic part mathrm h 0 and of a gaussian random potential v finally we consider the extension to the time dependent case

the itzykson zuber integral for u m n the itzykson zuber iz integral is computed for the superunitary group u m n as a consequence the nonzero correlations of superunitary matrices can be found

dyson s correlation functions and graded symmetry a new derivation of dyson s k level correlation functions of the gaussian unitary ensemble gue is given the method uses matrices with graded symmetry the number of integrations needed for the ensemble average becomes independent of the level number n for arbitrary level number n the k level correlation function is expressed as an integral involving the eigenvalues of a 2k 2k graded matrix the limit of infinitely many levels n is calculated by a simple saddle point approximation of this integral avoiding the introduction of hermite polynomials and oscillator wave functions

the k point random matrix kernels obtained from one point supermatrix models the k point correlation functions of the gaussian random matrix ensembles are certain determinants of functions which depend on only two arguments they are referred to as kernels since they are the building blocks of all correlations we show that the kernels are obtained for arbitrary level number directly from supermatrix models for one point functions more precisely the generating functions of the one point functions are equivalent to the kernels this is surprising because it implies that already the one point generating function holds essential information about the k point correlations this also establishes a link to the averaged ratios of spectral determinants i e of characteristic polynomials

mean field equations for spin models with orthogonal interaction matrices we study the metastable states in ising spin models with orthogonal interaction matrices we focus on three realizations of this model the random case and two non random cases i e the fully frustrated model on an infinite dimensional hypercube and the so called sine model we use the mean field or tap equations which we derive by resumming the high temperature expansion of the gibbs free energy in some special non random cases we can find the absolute minimum of the free energy for the random case we compute the average number of solutions to the tap equations we find that the configurational entropy or complexity is extensive in the range trsb t tm we also present an apparently unrelated replica calculation which reproduces the analytical expression for the total number of tap solutions

imaginary axis eigenvalues of matrix delay equations with a certain alternating coefficient structure abstract in this paper we examine the imaginary axis eigenvalues of matrix delay differential equations with coefficients alternating between hermitian and skew hermitian such as occurring in many mechanical systems we show that given this special structure and a certain sign condition the dimension of the eigenvalue problem is greatly reduced we also give some theorems on eigenvalue crossing directions in this context and show how reduction of dimension and crossing theorems work to provide insight on mechanical systems with time delay

stability robustness analysis of multiple time delayed systems using building block concept an intriguing perspective is presented in studying the stability robustness of systems with multiple independent and uncertain delays it is based on a holographic mapping which is implemented over the domain of the delays this mapping considerably alleviates the problem which is otherwise known to be notoriously complex it creates a dramatic reduction in the dimension of the problem from infinity to manageably small number ultimately the process is reduced to studying the problem within a finite dimensional cube with edges of length 2pi in the new domain what we call the building block in essence the mapping collapses the entire set of potential stability switching points onto a small upperbounded number of building hypersurfaces we further demonstrate that these building hypersurfaces can be implicitly defined and they are completely isolated within the above mentioned cube it is also shown that the exhaustive detection of these building hypersurfaces is necessary and sufficient in order to arrive at the complete stability robustness picture we seek as a consequence this concept yields a very practical and efficient procedure for the stability assessment of such systems this novel perspective serves very well for the preparatory steps of the authors earlier contribution in the area cluster treatment of characteristic roots ctcr we elaborate on this combination which forms the main contribution of the paper several example case studies are also provided

universal singularity at the closure of a gap in a random matrix theory we consider a hamiltonian h h 0 v in which h 0 is a given non random hermitian matrix and v is an n times n hermitian random matrix with a gaussian probability distribution we had shown before that dyson s universality of the short range correlations between energy levels holds at generic points of the spectrum independently of h 0 we consider here the case in which the spectrum of h 0 is such that there is a gap in the average density of eigenvalues of h which is thus split into two pieces when the spectrum of h 0 is tuned so that the gap closes a new class of universality appears for the energy correlations in the vicinity of this singular point