Autonomous reinforcement learning with experience replay
This paper considers the issues of efficiency and autonomy that are required to make reinforcement learning suitable for real-life control tasks. A real-time reinforcement learning algorithm is presented that repeatedly adjusts the control policy with the use of previously collected samples, and autonomously estimates the appropriate step-sizes for the learning updates. The algorithm is based on the actorcritic with experience replay whose step-sizes are determined on-line by an enhanced fixed point algorithm for on-line neural network training. Anexperimental study with simulated octopus arm and half-cheetah demonstrates the feasibility of the proposed algorithm to solve difficult learning control problems in an autonomous way within reasonably short time.
reinforcement learning actorcritic autonomous learning step-size estimation