Genre: Information/Explanation

A good way to understand reinforcement learning is to consider some of the examples and possible applications that have guided its development : A master chess player makes a move . The choice is informed both by planning---anticipating possible replies and counter-replies---and by immediate , intuitive judgments of the desirability of particular positions and moves . An adaptive controller adjusts parameters of a petroleum refinery's operation in real time . The controller optimizes the yield / cost / quality tradeoff based on specified marginal costs without sticking strictly to the set points originally suggested by human engineers . A gazelle calf struggles to its feet minutes after being born . Half an hour later it is running at 30 miles per hour . A mobile robot decides whether it should enter a new room in search of more trash to collect or start trying to find its way back to its battery recharging station . It makes its decision based on how quickly and easily it has been able to find the recharger in the past . Phil prepares his breakfast . When closely examined , even this apparently mundane activity reveals itself as a complex web of conditional behavior and interlocking goal-subgoal relationships : walking to the cupboard , opening it , selecting a cereal box , then reaching for , grasping , and retrieving the box . Other complex , tuned , interactive sequences of behavior are required to obtain a bowl , spoon , and milk jug . Each step involves a series of eye movements to obtain information and to guide reaching and locomotion . Rapid judgments are continually made about how to carry the objects or whether it is better to ferry some of them to the dining table before obtaining others . Each step is guided by goals , such as grasping a spoon , or getting to the refrigerator , and is in service of other goals , such as having the spoon to eat with once the cereal is prepared and of ultimately obtaining nourishment . These examples share features that are so basic that they are easy to overlook . All involve interaction between an active decision-making agent and its environment , within which the agent seeks to achieve a goal despite uncertainty about its the environment . The agent's actions are permitted to affect the future state of the environment ( e.g. , the next chess position , the level of reservoirs of the refinery , the next location of the robot ) , thereby affecting the options and opportunities available to the agent at later times . Correct choice requires taking into account indirect , delayed consequences of actions , and thus may require foresight or planning . At the same time , in all these examples the effects of actions can not be fully predicted , and so the agent must frequently monitor its environment and react appropriately . For example , Phil must watch the milk he pours into his cereal bowl to keep it from overflowing . All these examples involve goals that are explicit in the sense that the agent can judge progress toward its goal on the basis of what it can directly sense . The chess player knows whether or not he wins , the refinery controller knows how much petroleum is being produced , the mobile robot knows when its batteries run down , and Phil knows whether or not he is enjoying his breakfast . In all of these examples the agent can use its experience to improve its performance over time . The chess player refines the intuition he uses to evaluate positions , thereby improving his play ; the gazelle calf improves the efficiency with which it can run ; Phil learns to streamline his breakfast making . The knowledge the agent brings to the task at the start---either from previous experience with related tasks or built into it by design or evolution---influences what is useful or easy to learn , but interaction with the environment is essential for adjusting behavior to exploit specific features of the task .