# Atari early

Deepmind [announced](https://deepmind.com/blog/article/Agent57-Outperforming-the-human-Atari-benchmark) that their Agent57 beats the ‘human baseline’ at all 57 Atari games usually used as a benchmark. I think this is probably enough to resolve one of the predictions we had respondents make in our 2016 survey.

Our question was when it would be feasible to ‘outperform professional game testers on all Atari games using no game specific knowledge’.[^1^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-1-2351 "



<p>Full question wording:</p>



<p>How many years until you think the following AI tasks will be feasible with:</p>



<p>a small chance (10%)?
an even chance (50%)?
a high chance (90%)?</p>



<p>Let a task be ‘feasible’ if one of the best resourced labs could implement it in less than a year if they chose to. Ignore the question of whether they would choose to.</p>



[…]



<p>Outperform professional game testers on all Atari games using no game-specific knowledge. This includes games like Frostbite, which require planning to achieve sub-goals and have posed problems for deep Q-networks1,2.</p>



<p>1 Mnih et al. (2015). Human-level control through deep reinforcement learning
2 Lake et al. (2015). Building Machines That Learn and Think Like People</p>



<p>small chance (10%)<br> even chance (50%)<br> high chance (90%)") ‘Feasible’ was defined as meaning that one of the best resourced labs could do it in a year if they wanted to.

As I see it, there are four non-obvious things to resolve in determining whether this task has become feasible:

*   Did or could they outperform ‘professional game testers’?
*   Did or could they do it ‘with no game specific knowledge’?
*   Did or could they do it for ‘all Atari games’?
*   Is anything wrong with the result?

**I. Did or could they outperform ‘professional game testers’?**

It looks like yes, for at least for 49 of the games: the ‘human baseline’ seems to have come specifically from professional game testers.[^2^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-2-2351 "<a href="https://www.nature.com/articles/nature14236">This paper</a> seems to describe measuring the &#8216;human baseline&#8217;, in particular: &#8220;The professional human tester used the same emulator engine as the agents, and played under controlled conditions. The human tester was not allowed to pause, save or reload games. As in the original Atari 2600 environment, the emulator was run at 60 Hz and the audio output was disabled: as such, the sensory input was equated between human player and agents. The human performance is the average reward achieved from around 20 episodes of each game lasting a maximum of 5min each, following around 2 h of practice playing each game.&#8221;") What exactly the comparison was for the other games is less clear, but it sounds like what they mean by ‘human baseline’ is ‘professional game tester’, so probably the other games meet a similar standard.

**II. Did or could they do it with ‘no game specific knowledge’?**

It sounds like their system does not involve ‘game specific knowledge’, under a reasonable interpretation of that term.

**III. Did or could they do it for ‘all Atari games’?**

Agent57 only plays 57 [Atari 2600](https://en.wikipedia.org/wiki/Atari_2600) games, whereas [there are hundreds](https://en.wikipedia.org/wiki/List_of_Atari_2600_games) of Atari 2600 games (and [other](https://en.wikipedia.org/wiki/Atari_XEGS) [Atari](https://en.wikipedia.org/wiki/Atari_7800) [consoles](https://en.wikipedia.org/wiki/Atari_Jaguar) with presumably even more games).

Supposing that Atari57 is a longstanding benchmark including only 57 Atari games, it seems likely that the survey participants interpreted the question as about only those games. Or at least about all Atari 2600 games, rather than every game associated with the company Atari.

Interpreting it as written though, does Agent57’s success suggest that playing all Atari games is now feasible? My guess is yes, at least for Atari 2600 games.

Fifty-five of the fifty-seven games were proposed in [this paper](https://arxiv.org/pdf/1207.4708.pdf)[^3^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-3-2351 "Section 3.1.2, <a href="https://arxiv.org/pdf/1207.4708.pdf">https://arxiv.org/pdf/1207.4708.pdf</a>"), which describes how they chose fifty of them:

> Our testing set was constructed by choosing semi-randomly from the 381 games listed on Wikipedia \[http://en.wikipedia.org/wiki/List\_of\_Atari\_2600\_games (July 12, 2012)\] at the time of writing. Of these games, 123 games have their own Wikipedia page, have a single player mode, are not adult-themed or prototypes, and can be emulated in ALE. From this list, 50 games were chosen at random to form the test set.

The other five games in that paper were a ‘training set’, and I’m not sure where the other two came from, but as long as fifty of them were chosen fairly randomly, the provenance of the last seven doesn’t seem important.

My understanding is that none of the listed constraints should make the subset of games chosen particularly easy rather than random. So being able to play these games well suggests being able to play any Atari 2600 game well, without too much additional effort.

This might not be true if having chosen those games (about eight years ago), systems developed in the meantime are good for this particular set of games, but a different set of methods would have been needed had a different subset of games been chosen, to the extent that more than an additional year would be needed to close the gap now. My impression is that this isn’t very likely.

In sum, my guess is that respondents usually interpreted the ambiguous ‘all Atari games’ at least as narrowly as Atari 2600 games, and that a well resourced lab could now develop AI that played all Atari 2600 games within a year (e.g. plausibly DeepMind could already do that).

**IV. Is there anything else wrong with it?**

Not that I know of.

~

Given all this, I think it is more likely than not that this Atari task is feasible now. Which is interesting, because the [median 2016 survey response](https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/#Answers-9) put a 10% chance on it being feasible in five years, i.e. by 2021.[^4^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-4-2351 "Though note that only 19 participants answered the question about when there was a 10% chance. </p>



<p>We surveyed 352 machine learning researchers publishing at top conferences, asking each a random subset of many questions. Some of these questions were about when they expected thirty-two concrete AI tasks would become ‘feasible’. We asked each of those questions in two slightly different ways. The relevant Atari questions had 19 and 20 responses for the two wordings, only one of which gave an answer for 2021.") They more robustly put a median 50% chance on ten years out (2026).[^5^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-5-2351 "Half the time we asked about chances in N years, and half the time we asked about years until P probability, and people fairly consistently <a href="https://aiimpacts.org/chance-date-bias/">had earlier distributions</a> when asked the second way. Both methods yielded a 50% chance in ten years here, though later the distributions diverge, with a 90% chance in 15 years yet a 60% chance in 20 years. Note that small numbers of different respondents answered each question, so inconsistency is not a huge red flag, though the consistent inconsistency across many questions is highly suspicious.")

It’s exciting to start resolving expert predictions about early tasks so we know more about how to treat their later predictions about human-level science research and the obsolescence of all human labor for instance. But we should probably resolve a few more before reading much into it.

At a glance, some other tasks which we might be able to resolve soon:

*   The ‘reading Aloud’ task[^6^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-6-2351 "&#8216;Take a written passage and output a recording that can’t be distinguished from a voice actor, by an expert listener.&#8217;") [seems to be coming along](https://www.descript.com/lyrebird-ai?source=lyrebird) to my very non-expert ear, but I know almost nothing about it.
*   It seems like we are [close on Starcraft](https://aiimpacts.org/the-unexpected-difficulty-of-comparing-alphastar-to-humans/) though as far as I know the prediction hasn’t been exactly resolved as stated.[^7^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-7-2351 "Beat the best human Starcraft 2 players at least 50% of the time, given a video of the screen.<br>Starcraft 2 is a real time strategy game characterized by:<br><br>Continuous time play<br>Huge action space<br>Partial observability of enemies Long term strategic play, e.g. preparing for and then hiding surprise attacks.&#8217;")
*   Human-level Angry Birds play[^8^](https://aiimpacts.org/atari-early/#easy-footnote-bottom-8-2351 "</span>Play new levels of Angry Birds better than the best human players. Angry Birds is a game where players try to efficiently destroy 2D block towers with a catapult. For context, this is the goal of the IJCAI Angry Birds AI competition<sup>1</sup>.<br><br><sup>1 aibirds.org</sup>") was forecast for four years out with 50% chance, and [hasn’t happened](http://aibirds.org/past-competitions/2019-competition/man-vs-machine-challenge.html) yet. I note that the [contest website cites our survey as pressure to do better](http://aibirds.org/past-competitions/2019-competition/man-vs-machine-challenge.html), slightly complicating things.
*   AI that could ‘play poker well enough to win the World Series of Poker’ had a median 50% forecast in three years (i.e. 2019). In 2019, [a system beat elite professional players at six-player no-limit Texas hold’em](https://science.sciencemag.org/content/365/6456/885) [including Chris ‘Jesus’ Ferguson](https://www.sciencedaily.com/releases/2019/07/190711141343.htm) [2000 winner of World Series main event](https://en.wikipedia.org/wiki/List_of_World_Series_of_Poker_Main_Event_champions). However World Series contains several versions of Poker, so it isn’t clear that AI could actually win the World Series. I’m not familiar enough with Poker to say whether any of the differences between Texas Hold’em, Omaha Hold’em and Seven Card Stud should make the latter two difficult if the first is now feasible.

_By Katja Grace_

_Thanks to Rick Korzekwa, Jacob Hilton and Daniel Filan for answering many questions._

Notes
-----