An Introduction to Reinforcement Learning
By: Anushka Upadhyaya
Tags: Reinforcement Learning

“The development of full artificial intelligence could spell the end of the human race….It would take off on its own, and re-design itself at an ever-increasing rate. Humans, who are limited by slow biological evolution, couldn’t compete, and would be superseded.” — Stephen Hawking

From the amazing results in Vintage Atari Games, Deep minds victory with AlphaGo, stunning breakthroughs in robotic arm manipulation and even beating professional players at Dota-v1, the field of Reinforcement Learning (RL) has literally exploded in recent years. RL is really hot!

Ever since the impressive breakthrough on the imagenet classification challenge in 2012 the success of supervised deep learning has continued to pile up and people from many different backgrounds have started using deep neural nets to solve a wide range of new tasks including how to learn intelligent behavior in complex dynamic environments.

Examples of RL in the wild. From Left to Right. Deep Q Learning network playing ATARI, AlphaGo, Berkeley robot Stacking Legos, physically-simulated quadruped leaping over Terrain.

Now, imagine when you were a child, how curious you were to learn anything and everything and challenge your own limits. In the summer vacations, when you started to learn swimming, how you were a teeny-tiny child who used to hold mum’s hand initially and was scared of the depths of water. Then you upgraded yourself to wearing a rescue tube around your waist and with baby steps your fear of water started vanishing. Next, your instructor must have taught you to hold the bars at the corners of the swimming pool and start peddling your legs and with that, you learned how to swim and in no time you mastered all kinds of strokes from front-stroke to fish-stroke.

Now, let’s introduce this beautiful childhood memory of yours into the depths of Technology specifically in RL- the art of Action and Reward. The world of Swimming — acting as an environment, you as a child-Agent perform certain actions, upgrading from holding mum’s hand in waters, to rescue tubes, then to practicing pedaling with holding bars to learning strokes and finally jumping into the deepest of waters. With every action you performed, you learned certain techniques of swimming, and with good practice, learning from failed actions, you eventually used your mind and learned how to perfect each swimming skill. Do you also remember how this water phobia of yours vanished with each action performed giving you a certain reward in the form of fearlessness and pleasure.

If you can relate to all of this, Congratulations. You now know what Reinforcement Learning is.

“RL is a part of Machine learning where an agent is put in an environment and he learns to behave in this environment by performing certain actions and observing the rewards which it gets from those actions.”

So, the question is why is Reinforcement Learning such a hot topic?

Well, the answer to this lies in the most popular application of Deep Reinforcement Learning is of Google’s Deepmind and its robot named AlphaGo/AlphaZero which has made huge headlines in recent years when it beat the world champion in Go and various other Chess/shogi engines. On the other hand, RL is so effective for problems when it is unknown which action is best in each state, and in most problems is quite hard to traceback and untangle which action led to which result.