Patent ID: 8639556
Filing Date: 2014-01-28
Classification: G06Q

Abstract:
1. A method for determining an optimal action in consideration of risk, the method comprising the steps of: (a) selecting, with a processing device, a state from possible states in a current term, wherein said current term belongs to a time horizon; (b) selecting, with a processing device, an action from at least one action candidate that can be executed in a selected state; (c) calculating, with a processing device, a probability distribution of an evaluation value for a selected action, wherein said evaluation value corresponds to a reward obtained in said current term and an optimal value for next term; (d) calculating, with a processing device, a risk measure using said probability distribution of said evaluation value; (e) determining, with a processing device, a weighting function conforming to a preference by taking said risk measure into consideration; (f) calculating, with a processing device, a value measure of said selected action; (g) repeating steps (b) through (f) for all other said action candidates that can be executed in said selected state; and (h) comparing, with a processing device, said value measures of said selected actions in order to determine an optimal action for said selected state; wherein said processing device is configured to be able to access: a transition probability storing unit that stores, for each of said possible states, a transition probability of transitioning to said state when said action is executed in said state; and a reward parameter storing unit that stores, for each of said possible states, at least one parameter indicative of said probability distribution of said reward obtained when transition to said state occurs as a result of executing said action in said state, wherein said computer device generates, with reference to said transition probability and said at least one parameter, a graph data that comprises: node data representing said possible states in said time horizon; directed edge data representing transitions from said possible states in said current term to possible states for said next term; and attribute data representing at least one said action that can be executed in each of said possible states in said current term, said transitional probability of transitioning to each target state when each of at least one said action is executed, and said probability distribution of said reward that is obtained when said action is executed.