Patent ID: 8660670

Claim:
A controller for controlling operation of a machine operating in an operating environment, the machine being responsive to control signals to perform a plurality of actions, the controller receiving signals indicative of parameters relating to the operating environment and sufficient to determine a score relating to an outcome of operation of the machine, the controller comprising: (a) at least one output for providing control signals to the machine to perform selected actions; (b) at least one input for receiving signals indicative of parameters relating to the operating environment; (c) a data storage device containing a historical database including: at least one sequence of parameters relating to the operating environment; corresponding actions taken; and corresponding scores relating to outcomes of operation of the machine; and (d) a processing system including at least one processor, said processing system being in data communication with said data storage device, said at least one output and said at least one input, said processing system being configured to: (i) maintaining a list of episodes, corresponding to sequences identified within the historical database, which satisfy a relevance criteria relative to a current sequence of parameters, said relevance criteria employing a relevance function corresponding to an overall measure of similarity between the current sequence of parameters and the episode derived by combining a plurality of similarity functions, values of said relevance function for each episode in said list being updated based on recently input parameters; (ii) perform a randomized selection between a plurality of actions or sequences of actions, at least one of said actions or sequences of actions being derived from an episode in said list that had a favorable outcome of operation of the machine; and (iii) outputting at least one control signal to the machine via said at least one output to perform the selected action or sequence of actions.