Patent ID: 7403904

Claim:
A method for sequential decision making for customer relationship management, comprising: providing customer data comprising stimulus-response history data for a population of customers, said stimulus-response history data being derived from event data for said customers; automatically generating actionable rules for optimizing a sequence of decisions over a period of time based on said stimulus-response history data; estimating a value function using batch reinforcement learning with function approximation, said function approximation representing the value function as a function of state features and actions, and said estimating a value function using batch reinforcement learning with function approximation comprising: estimating a function approximation of the value function of a Markov Decision Process underlying said stimulus-response history data for said population of customers; and iteratively applying a regression model to training data comprising sequences of states, actions and rewards resulting for said population of customers, and updating in each iteration a target reward value for each state-action pair; and transforming an output of a value function estimation into said actionable rules, the rules specifying what actions to take given a set of feature values corresponding to a customer, and the action taken corresponding to an action having an approximate maximum value according to said value function for the given set of feature values.