Patent Document ID: 7966276
Application ID: 11884565
Patent Flag: 1

Claim One:
1. A controller, for controlling a system on the basis of measurement data received from a plurality of sensors indicative of a state of the system, wherein the controller comprises: a system model and corresponding measurement models for the plurality of sensors of the system; a stochastic estimator for receiving measurement data from the plurality of sensors and for generating, with reference to the system model, a plurality of samples each representative of the state of the system; a rule set corresponding to the system model, defining, for each of a plurality of possible samples representing possible states of the system, information defining an action to be carried out in the system; and an action selector, for receiving an output of the stochastic estimator and for selecting, with reference to the rule set, information defining one or more corresponding actions to be performed in the system; wherein the controller is configured to: (i) construct an initial partially observed Markov decision process (POMDP) model representing the dynamics of the system to be controlled, wherein the POMDP model comprises a representation of the states of the system, a measurement model, one or more control actions, and measures of benefit likely to arise from the selection of particular control actions; (ii) transform the initial POMDP model into a subsidiary Markov decision process (MDP) model, comprising generating a sample state space representation for the subsidiary model, and generating an initial probabilistic system model and control rule set using the sample state representation; and (iii) use observations of the system and of the environment by a plurality of sensors to update the control rule set and the probabilistic system model of the subsidiary MDP based upon the observed effects of selected control actions and with reference to the measures of the benefit system.