How AI Learns to Play Games
Tags: machinelearning, reinforcementlearnin, ai
Published: 2019-06-03 12:53:45

---
title: How AI Learns to Play Games
published: true
description: A brief introduction to the reinforcement learning paradigm, from the perspective of game agents.
tags: MachineLearning, ReinforcementLearnin, AI
canonical_url: https://reinforcementlearning4.fun/2019/06/03/how-ai-learns-play-games/
cover_image: https://launchyourintelligentapphome.files.wordpress.com/2019/06/1453929309-go-board-hoge-rielen-belgiumedit-fcb981.jpg?resize=900%2C900
---

<!-- wp:paragraph -->
<p>Over the past few years, we've seen computer programs winning games which we believe humans were unbeatable. This belief held considering this games had so many possible moves for a given position that would be impossible to computer programs calculate all of then and choose the best ones. However, in 1997 the world witnessed what otherwise was considered impossible: the <a rel="noreferrer noopener" aria-label="IBM Deep Blue (opens in a new tab)" href="https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)" target="_blank">IBM Deep Blue</a> supercomputer won a six game chess match against <a rel="noreferrer noopener" aria-label="Gary Kasparov (opens in a new tab)" href="https://en.wikipedia.org/wiki/Garry_Kasparov" target="_blank">Gary Kasparov</a>, the world champion of that time, by 3.5 - 2.5. Such victory would only be achieved again when DeepMind's <a rel="noreferrer noopener" aria-label="AlphaGo (opens in a new tab)" href="https://en.wikipedia.org/wiki/AlphaGo" target="_blank">AlphaGo</a> won a five game Go match against <a href="https://en.wikipedia.org/wiki/Lee_Sedol" target="_blank" rel="noreferrer noopener" aria-label="Lee Sedol (opens in a new tab)">Lee Sedol</a>, 18 times world champion, by a 4-1 score.</p>
<!-- /wp:paragraph -->

<!-- wp:more -->
<!--more-->
<!-- /wp:more -->

<!-- wp:image {"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://img.purch.com/w/660/aHR0cDovL3d3dy5saXZlc2NpZW5jZS5jb20vaW1hZ2VzL2kvMDAwLzA5Mi8xNTUvb3JpZ2luYWwvZ2Fycnkta2FzcGFyb3YtaGFuZC1zaGFrZS5qcGVn" alt=""/><figcaption>World Chess champion Garry Kasparov barely acknowledges the handshake from Dr. C.J. Tan head of the IBM Deep Blue computer team which defeated Kasparov in the six-game series that ended on May 11, 1997.<br>Credit: Roger Celestin/Newscom</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>The IBM Deep Blue team relied mostly on brute force and computation power as their strategy to win the matches. Equipped with thousands of VLSI chips, specially designed to evaluate chess positions, the Deep Blue was capable to evaluate 200 millions of positions per second. Additionally, the Deep Blue software had a database of more than 4,000 openings and more than 700,000 games from grandmasters.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":66,"align":"center","width":533,"height":355} -->
<div class="wp-block-image"><figure class="aligncenter is-resized"><img src="https://launchyourintelligentapphome.files.wordpress.com/2019/05/go-lee-sedol.jpg" alt="" class="wp-image-66" width="533" height="355"/><figcaption>Lee Se-dol played a five-game match against a computer program developed by a Google, AlphaGo.  (Photo by Google via Getty Images)</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>However, a brute-force strategy is not feasible to play Go, due to the much higher number of possible movies. Instead, the DeepMind had to make AlphaGo more intelligent in order to prevent all those calculations. The version that played against Lee Sedol combined machine learning and tree search techniques, as well as few heuristics to detect specific game patterns as the <a href="https://en.wikipedia.org/wiki/List_of_Go_terms#Nakade" target="_blank" rel="noreferrer noopener" aria-label="nakade (opens in a new tab)">nakade</a>, for example. Initially the algorithm was trained with games played by humans, and after reaching some proficiency, it started to learn by playing against other instances of itself. After winning the match against Sedol, the DeepMind team continued improving AlphaGo, and its latest version called AlphaZero was trained only by playing against itself. </p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>After the success achieved with AlphaGo, <a rel="noreferrer noopener" aria-label="DeepMind (opens in a new tab)" href="https://deepmind.com/blog/" target="_blank">DeepMind</a>, <a rel="noreferrer noopener" aria-label="OpenAI (opens in a new tab)" href="https://openai.com/blog/" target="_blank">OpenAI</a> and other research companies started to work in even more complex games, like <a rel="noreferrer noopener" aria-label="Starcraft2 (opens in a new tab)" href="https://twitter.com/rodolfo_mmendes/status/1134993972420644864" target="_blank">Starcraft2</a> and <a href="https://twitter.com/rodolfo_mmendes/status/1134062075154767872" target="_blank" rel="noreferrer noopener" aria-label="Dota2 (opens in a new tab)">Dota2</a>. So, how can computer programs can learn to play this games, with professional proficiency? These incredible achievements were possible applying a powerful artificial intelligence technique called <strong>reinforcement learning</strong>.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2> What is reinforcement learning?</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Reinforcement learning is an artificial intelligence paradigm in which an intelligent agent learns to execute tasks by trial and error, interacting with the environment. Instead of learning from a labeled dataset, like in supervised learning, in reinforcement learning the agent obtain information about the environment, performs an action based on this information and then receives a reinforcement signal to indicate if the action was correct.</p>
<!-- /wp:paragraph -->

<!-- wp:image {"id":95,"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://launchyourintelligentapphome.files.wordpress.com/2019/05/action-reward.png" alt="" class="wp-image-95"/><figcaption>The agent performs an action, and receives an observation and a reward from the environment.</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>For example, suppose we want to teach a robot to walk in a room without hitting the walls. Using different types of sensors we can collect information about the environment, like the distance between the robot and the walls and the speed of the robot in different directions. In order to indicate to the agent that it is performing the task correctly, we must be able to encode reward signals to be read together with the states. We could do this by rewarding a step in any direction with a +1 signal and penalizing the agent with a -1 signal when a collision is detected. So, put in a very simple way, what a reinforcement learning agent does is to record the rewards received by the actions so it can learn which are the best ones to choose given its current state.</p>
<!-- /wp:paragraph -->

<!-- wp:heading -->
<h2>Conclusion</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Along the decades, Reinforcement Learning has been shown a very powerful machine learning paradigm and has been applied to create intelligent robots, chatbots, trading bots, autonomous vehicles and other areas that require automation and optimization. Complex games like Chess, Go and more recently Starcraft2 and Dota2 have been used by researchers as benchmarks to establish the current state-of-the-art algorithms for Reinforcement Learning.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>If you're interested in going deep in Reinforcement Learning, check our post <a href="https://dev.to/rodolfomendes/top-5-free-courses-in-reinforcement-learning-18j0">Top 5 Courses in Reinforcement Learning</a>, to have a list of 5 free courses offered by the top of mind companies and universities. In future posts, we're going to explore the elementary concepts of Reinforcement Learning through the Frozen Lake game example.</p>

<p>
Original post:
<a href="https://reinforcementlearning4.fun/2019/06/03/how-ai-learns-play-games/" target="_blank">How AI Learns to Play Games</a></p>