Patent Document ID: 8412649
Application ID: 12649244
Patent Status: 1

Claim One:
1. A method for converting a continuous system to a Markov decision process (MDP), wherein the continuous system is dynamical and the MDP has discrete states, comprising the steps of: selecting a predetermined number of continuous states of the continuous system, wherein each continuous state corresponds to one discrete state of the MDP; and applying Delaunay triangulation to the continuous states to produce a set of triangles, wherein vertices of each triangle represent the continuous states, and further comprising for each discrete state; determining a next discrete state y=ƒ(x,a), wherein x represents the continuous state corresponding to the discrete state, a is a control action, and ƒ is a non-linear transition function for the continuous; identifying a particular triangle containing the next discrete state y; and expressing the next discrete state y as probabilities of transitioning to the discrete states corresponding to the continuous states x represented by the vertices of the particular triangle, where identifying the particular triangle is performed by means of forming a d×d difference matrix E, such that column j of the difference matrix E contains a differences v m,j −v m.d+1 , for j=1, d, between an j th vertex v m.j of triangle m and a d+1 st vertex q=v m.d+1 of the triangle m, and solving a linear system of equations Ec=(y−q), for a vector c, and wherein the triangle m is identified as containing y if all c j >0, j=1, d, and c d+1 =1−Σ j=1 D c j >0, wherein the steps are performed in a processor.