r max a general polynomial time algorithm for near optimal reinforcement learning r max is a very simple model based reinforcement learning algorithm which can attain near optimal average reward in polynomial time in r max the agent always maintains a complete but possibly inaccurate model of its environment and acts based on the optimal policy derived from this model the model is initialized in an optimistic fashion all actions in all states return the maximal possible reward hence the name during execution it is updated based on the agent s observations r max improves upon several previous algorithms 1 it is simpler and more general than kearns and singh s e3 algorithm covering zero sum stochastic games 2 it has a built in mechanism for resolving the exploration vs exploitation dilemma 3 it formally justifies the optimism under uncertainty bias used in many rl algorithms 4 it is simpler more general and more efficient than brafman and tennenholtz s lsg algorithm for learning in single controller stochastic games 5 it generalizes the algorithm by monderer and tennenholtz for learning in repeated games 6 it is the only algorithm for learning in repeated games to date which is provably efficient considerably improving and simplifying previous algorithms by banos and by megiddo

nonmyopic multiaspect sensing with partially observable markov decision processes we consider the problem of sensing a concealed or distant target by interrogation from multiple sensors situated on a single platform the available actions that may be taken are selection of the next relative target platform orientation and the next sensor to be deployed the target is modeled in terms of a set of states each state representing a contiguous set of target sensor orientations over which the scattering physics is relatively stationary the sequence of states sampled at multiple target sensor orientations may be modeled as a markov process the sensor only has access to the scattered fields without knowledge of the particular state being sampled and therefore the problem is modeled as a partially observable markov decision process pomdp the pomdp yields a policy in which the belief state at any point is mapped to a corresponding action the nonmyopic policy is compared to an approximate myopic approach with example results presented for measured underwater acoustic scattering data

phase transitions in an exactly soluble one dimensional exclusion process we consider an exclusion process with particles injected with rate at the origin and removed with rate at the right boundary of a one dimensional chain of sites the particles are allowed to hop onto unoccupied sites to the right only for the special case of 1 the model was solved previously by derridaet al here we extend the solution to general the phase diagram obtained from our exact solution differs from the one predicted by the mean field approximation

learning robust similarity measures for 3d partial shape retrieval in this paper we propose a novel approach to learning robust ground distance functions of the earth mover s distance to make it appropriate for quantifying the partial similarity between two feature sets first we define the ground distance as a monotonic transformation of commonly used feature to feature base distance or similarity measures so that in computing the earth mover s distance the algorithm could better turn its focus on the feature pairs that are correctly matched while being less affected by irrelevant ones as a result the proposed method is especially suited for 3d partial shape retrieval where occlusion and clutter are serious problems we prove that when the transformation satisfies certain conditions the metric property of the base distance is sufficient to guarantee the ground distance is a metric and so is the earth mover s distance which makes fast shape retrieval on large databases technically possible second we propose a discriminative learning framework to optimize the transformation function based on the real adaboost algorithm the optimization is performed in the space of the piecewise constant approximations of the transformation without making any parametric assumption finally extensive experiments on 3d partial shape retrieval convincingly demonstrate the effectiveness of the proposed techniques

self organized critical forest fire model mean field theory and simulation results in 1 to 6 dimenisons we argue that the critical forest fire model ffm introduced by drossel and schwabl phys rev lett 69 1629 1992 is a critical branching process in the mean field approximation where the number s of trees burned in a forest fire is power law distributed with exponent s 5 2 the mean field model of the ffm in finite dimension is the percolation model and as in percolation the upper critical dimension is 6 simulations show that the ffm becomes increasingly percolationlike with increasing dimension d and is within error bars eully consistent with the percolation results when d 3

the earth mover s distance as a metric for image retrieval we investigate the properties of a metric between two distributions the earth mover s distance emd for content based image retrieval the emd is based on the minimal cost that must be paid to transform one distribution into the other in a precise sense and was first proposed for certain vision problems by peleg werman and rom for image retrieval we combine this idea with a representation scheme for distributions that is based on vector quantization this combination leads to an image comparison framework that often accounts for perceptual similarity better than other previously proposed methods the emd is based on a solution to the transportation problem from linear optimization for which efficient algorithms are available and also allows naturally for partial matching it is more robust than histogram matching techniques in that it can operate on variable length representations of the distributions that avoid quantization and other binning problems typical of histograms when used to compare distributions with the same overall mass the emd is a true metric in this paper we focus on applications to color and texture and we compare the retrieval performance of the emd with that of other distances

non abelian symmetries of stochastic processes derivation of correlation functions for random vertex models and disordered interacting particle systems we consider systems of particles hopping stochastically on d dimensional lattices with space dependent probabilities we map the master equation onto an evolution equation in a fock space where the dynamics are given by a quantum hamiltonian continuous time or a transfer matrix discrete time using non abelian symmetries of these operators we derive duality relations expressing the time evolution of a given initial configuration in terms of correlation functions of simpler dual processes particularly simple results are obtained for the time evolution of the density profile as a special case we show that for any su 2 symmetric system the two point and three point density correlation functions in the n particle steady state can be computed from the probability distribution of a single particle moving in the same environment we apply our results to various models among them partial exclusion a simple diffusion reaction system and the two dimensional six vertex model with space dependent vertex weights for a random distribution of the vertex weights one obtains a version of the random barrier model describing diffusion of particles in disordered media we derive exact expressions for the averaged two point density correlation functions in the presence of weak correlated disorder

orientation of particle attachment and local isotropy in diffusion limited aggregation dla we simulate 50 off lattice diffusion limited aggregation dla clusters of one million particles each the probability distribution of the angle of attachment of arriving particles with respect to the local radial direction is obtained numerically for increasing cluster size n the distribution crosses over extremely accurately to a cosine with an amplitude which decreases towards zero as a power law in n from this specific viewpoint asymptotically large dla clusters are locally isotropic

multi view 3d object retrieval with deep embedding network in multi view 3d object retrieval each object is characterized by a group of 2d images captured from different views rather than using hand crafted features in this paper we take advantage of the strong discriminative power of convolutional neural network to learn an effective 3d object representation tailored for this retrieval task specifically we propose a deep embedding network jointly supervised by classification loss and triplet loss to map the high dimensional image space into a low dimensional feature space where the euclidean distance of features directly corresponds to the semantic similarity of images by effectively reducing the intra class variations while increasing the inter class ones of the input images the network guarantees that similar images are closer than dissimilar ones in the learned feature space besides we investigate the effectiveness of deep features extracted from different layers of the embedding network extensively and find that an efficient 3d object representation should be a tradeoff between global semantic information and discriminative local characteristics then with the set of deep features extracted from different views we can generate a comprehensive description for each 3d object and formulate the multi view 3d object retrieval as a set to set matching problem extensive experiments on shrec 15 data set demonstrate the superiority of our proposed method over the previous state of the art approaches with over 12 performance improvement

exact results for the one dimensional self organized critical forest fire model we present the analytic solution of the self organized critical soc forest fire model in one dimension proving soc in systems without conservation laws by analytic means under the condition that the system is in the steady state and very close to the critical point we calculate the probability that a string of ital n neighboring sites is occupied by a given configuration of trees the critical exponent describing the size distribution of forest clusters is exactly tau 2 and does not change under certain changes of the model rules computer simulations confirm the analytic results

using spin images for efficient object recognition in cluttered 3d scenes we present a 3d shape based object recognition system for simultaneous recognition of multiple objects in scenes containing clutter and occlusion recognition is based on matching surfaces by matching points using the spin image representation the spin image is a data level shape descriptor that is used to match surfaces represented as surface meshes we present a compression scheme for spin images that results in efficient multiple object recognition which we verify with results showing the simultaneous recognition of multiple objects from a library of 20 models furthermore we demonstrate the robust performance of recognition in the presence of clutter and occlusion through analysis of recognition trials on 100 scenes

an efficient earth mover s distance algorithm for robust histogram comparison we propose emd l1 a fast and exact algorithm for computing the earth mover s distance emd between a pair of histograms the efficiency of the new algorithm enables its application to problems that were previously prohibitive due to high time complexities the proposed emd l1 significantly simplifies the original linear programming formulation of emd exploiting the l1 metric structure the number of unknown variables in emd l1 is reduced to o n from o n2 of the original emd for a histogram with n bins in addition the number of constraints is reduced by half and the objective function of the linear program is simplified formally without any approximation we prove that the emd l1 formulation is equivalent to the original emd with a l1 ground distance to perform the emd l1 computation we propose an efficient tree based algorithm tree emd tree emd exploits the fact that a basic feasible solution of the simplex algorithm based solver forms a spanning tree when we interpret emd l1 as a network flow optimization problem we empirically show that this new algorithm has an average time complexity of o n2 which significantly improves the best reported supercubic complexity of the original emd the accuracy of the proposed methods is evaluated by experiments for two computation intensive problems shape recognition and interest point matching using multidimensional histogram based local features for shape recognition emd l1 is applied to compare shape contexts on the widely tested mpeg7 shape data set as well as an articulated shape data set for interest point matching sift shape context and spin image are tested on both synthetic and real image pairs with large geometrical deformation illumination change and heavy intensity noise the results demonstrate that our emd l1 based solutions outperform previously reported state of the art features and distance measures in solving the two tasks

shapeme histogram projection and matching for partial object recognition histograms of shape signature or prototypical shapes called shapemes have been used effectively in previous work for 2d 3d shape matching and recognition we extend the idea of shapeme histogram to recognize partially observed query objects from a database of complete model objects we propose representing each model object as a collection of shapeme histograms and match the query histogram to this representation in two steps 1 compute a constrained projection of the query histogram onto the subspace spanned by all the shapeme histograms of the model and 2 compute a match measure between the query histogram and the projection the first step is formulated as a constrained optimization problem that is solved by a sampling algorithm the second step is formulated under a bayesian framework where an implicit feature selection process is conducted to improve the discrimination capability of shapeme histograms results of matching partially viewed range objects with a 243 model database demonstrate better performance than the original shapeme histogram matching algorithm and other approaches

hidden markov processes an overview of statistical and information theoretic aspects of hidden markov processes hmps is presented an hmp is a discrete time finite state homogeneous markov chain observed through a discrete time memoryless invariant channel in recent years the work of baum and petrie 1966 on finite state finite alphabet hmps was expanded to hmps with finite as well as continuous state spaces and a general alphabet in particular statistical properties and ergodic theorems for relative entropy densities of hmps were developed consistency and asymptotic normality of the maximum likelihood ml parameter estimator were proved under some mild conditions similar results were established for switching autoregressive processes these processes generalize hmps new algorithms were developed for estimating the state parameter and order of an hmp for universal coding and classification of hmps and for universal decoding of hidden markov channels these and other related topics are reviewed

doi peliti path integral methods for stochastic systems with partial exclusion abstract doi peliti methods are developed for stochastic models with finite maximum occupation numbers per site we provide a generalized framework for the different fock spaces reported in the literature paragrassmannian techniques are then utilized to construct path integral formulations of factorial moments we show that for many models of interest a magnus expansion is required to construct a suitable action meaning actions containing a finite number of terms are not always feasible however for such systems perturbative techniques are still viable and for some examples including carrying capacity population dynamics and diffusion with partial exclusion the expansions are exactly summable

the pyramid match kernel efficient learning with sets of features in numerous domains it is useful to represent a single example by the set of the local features or parts that comprise it however this representation poses a challenge to many conventional machine learning techniques since sets may vary in cardinality and elements lack a meaningful ordering kernel methods can learn complex functions but a kernel over unordered set inputs must somehow solve for correspondences generally a computationally expensive task that becomes impractical for large set sizes we present a new fast kernel function called the pyramid match that measures partial match similarity in time linear in the number of features the pyramid match maps unordered feature sets to multi resolution histograms and computes a weighted histogram intersection in order to find implicit correspondences based on the finest resolution histogram cell where a matched pair first appears we show the pyramid match yields a mercer kernel and we prove bounds on its error relative to the optimal partial matching cost we demonstrate our algorithm on both classification and regression tasks including object recognition 3 d human pose inference and time of publication estimation for documents and we show that the proposed method is accurate and significantly more efficient than current approaches

shape matching and object recognition using shape contexts we present a novel approach to measuring similarity between shapes and exploit it for object recognition in our framework the measurement of similarity is preceded by 1 solving for correspondences between points on the two shapes 2 using the correspondences to estimate an aligning transform in order to solve the correspondence problem we attach a descriptor the shape context to each point the shape context at a reference point captures the distribution of the remaining points relative to it thus offering a globally discriminative characterization corresponding points on two similar shapes will have similar shape contexts enabling us to solve for correspondences as an optimal assignment problem given the point correspondences we estimate the transformation that best aligns the two shapes regularized thin plate splines provide a flexible class of transformation maps for this purpose the dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points together with a term measuring the magnitude of the aligning transform we treat recognition in a nearest neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image results are presented for silhouettes trademarks handwritten digits and the coil data set

fluctuation effects and multiscaling of the reaction diffusion front for a b to oe we consider the properties of the diffusion controlled reaction a b to oe in the steady state where fixed currents of a and b particles are maintained at opposite edges of the system using renormalization group methods we explicitly calculate the asymptotic forms of the reaction front and particle densities as expansions in jd 1 mod x mod d 1 1 where j are the equal applied currents and d the equal diffusion constants for the asymptotic densities of the minority species we find in addition to the expected exponential decay fluctuation induced power law tails which for d 2 we find similar power laws with mu d 3 but with non universal coefficients logarithmic corrections occur in d 2 these results imply that in the time dependent case with segregated initial conditions the moments integral mod x mod qr x t dx fail to satisfy simple scaling for q mu 1 finally it is shown that the fluctuation induced wandering of the position of the reaction front centre may be neglected for large enough systems

analytical approach to traffic jams the traffic flow problem in two dimensions is formulated as a three state model on a square lattice in terms of pauli operators using a fock space representation of the master equation we get the liouvillian for the problem with asymmetric exclusion three different realizations symmetric right before left model and an exchange model will be analysed within the mean field approximation mfa the resulting kinetic equations for the average occupation number of cars in the upward and sideward directions are coupled the average velocity can be calculated in mfa it results in a jamming transition that depends on the total concentration of cars

fast kawasaki spin exchange limit of spin facilitated kinetic ising models we study a fock space operator technique for describing the stochastic kinetics of a spin facilitated kinetic ising model we focus in particular on the diffusion fast kawasaki exchange limit in which the kinetics can be described by a single mean field evolution equation we derive some general criteria for the approximative validity of mean field theory for the case of a nondiverging diffusion coefficient of the local spin states

synchronization of stochastic oscillations due to long range diffusion we investigate the effect of long range diffusive mixing on stochastic processes taking place on low dimensional catalytic supports as a working example the cyclic lattice lotka volterra llv model is used which is conservative at the mean field level and demonstrates fractal patterns and local oscillations when realized on low dimensional lattice supports we show that the local oscillations are synchronized when a weak long range diffusive process is added to llv and global oscillations of limit cycle type emerge this phenomenon is demonstrated as a nonequilibrium phase transition and takes place when the mixing to reaction rate p order parameter is above a critical point p c the value of the critical point is shown to depend on the kinetic parameters the global oscillations in this case emerge as a result of phase synchronization between local oscillations on sublattices

self organized critical forest fire model a forest fire model is introduced which contains a lightning probability f this leads to a self organized critical state in the limit f ensuremath rightarrow 0 provided that the time scales of tree growth and burning down of forest clusters are separated we derive scaling laws and calculate all critical exponents the values of the critical exponents are confirmed by computer simulations for a two dimensional system we show that the forest density in the critical state assumes its minimum possible value i e that energy dissipation is maximum

an exact solution of a one dimensional asymmetric exclusion model with open boundaries a simple asymmetric exclusion model with open boundaries is solved exactly in one dimension the exact solution is obtained by deriving a recursion relation for the steady state if the steady state is known for all system sizes less thann then our equation 8 gives the steady state for sizen using this recursion we obtain closed expressions 48 for the average occupations of all sites the results are compared to the predictions of a mean field theory in particular for infinitely large systems the effect of the boundary decays as the distance to the power 1 2 instead of the inverse of the distance as predicted by the mean field theory

three state model and chemical reactions a three state model is formulated in terms of raising and lowering operators as an application cyclic chemical reactions with three different species in the solid state are studied the evolution equations for the densities and for the correlation functions can be derived starting from a master equation in a generalized fock space representation on a lattice such a description guarantees the consideration of the excluded volume effect in contrast to the classical kinetic behaviour the system offers stable and unstable regimes depending on the averaged composition ratio