Patent Document ID: 7478073
Application ID: 10861146
Patent Flag: 1

Claim One:
1. A computer readable medium encoded with computer program instructions which cause a processor to execute a method of generating control signals for controlling action of a sensory-motor apparatus, comprising: determining the value (S(t)) of a set of one or more sensory variables (S) representative of information about an environment of said sensory-motor apparatus; outputting a set of one or more control signals (M(t)) configured to control action of said sensory-motor apparatus; calculating a respective reward (R(v,t)) associated with each of a plurality of candidate values (M Cand ) that can be taken by said set of control signals (M(t)), the calculating being adapted to evaluate rewards by calculating a function based on a set of one or more motivational variables (mot i ), said set of motivational variables including at least one task-independent internal motivational variable that is independent of the type of sensory-motor apparatus being controlled, the value of each motivational variable being derived based on at least one sensory-motor variable of a set of sensory-motor variables (SM) comprising said set of sensory variables (S) and a set of action variables (M) representative of the behavior of said sensory-motor apparatus; and deciding, which value should be taken by said set of control signals (M(t)), by analyzing the reward values calculated for respective different values (M Cand ) that can be taken by said set of control signals; wherein the calculating of respective rewards comprises: performing a history-dependent calculation to calculate the value of said at least one task-independent motivational variable (mot i ), said performing step comprising either: a) applying a history-dependent function (g) to at least one sensory-motor variable of said set, said history-dependent function (g) depending on one or more time-varying internal parameters in the computation of the at least one motivational variable (moti), or b) applying a function (f) to values (SM(t),SM(t−1)) taken at different times by at least one sensory-motor variable of said set, predicting, for each of said plurality of candidate values of said control signals (M), values of sensory-motor variables (SM) and of said set of motivational variables (mot i ) at a series of future instants, and calculating, for each of the plurality of candidate values (M Cand ), a reward value (R MCand ) which comprises the sum of a series of expected future rewards corresponding to said predicted values of said set of motivational variables at said series of future instants, wherein each expected reward is evaluated by calculating a function (OR) based on said at least one task-independent internal motivational variable (mot i ); and wherein the outputting step comprises outputting the set of control signals chosen in the deciding step based on said reward values calculated in the calculating step.