# A Survey of Planning and Learning in Games
## 3 Learning in Games
### Learning in Commercial Games  
Despite the great success and many achievements of learning in academia, the gaming industry has remained somewhat reluctant to adopt some of these techniques (e.g., DL and DRL), particularly when it comes to modeling agent behaviors. One possible explanation for this may be the lack of expertise of the game designers/programmers concerning some of the techniques developed in academia (e.g., DRL, DL, EAs, PGMs). Budget or time constraints are another possible explanation. In fact, it is estimated that only 20% of all computing resources are available to develop the game AI and, of these, a large portion is spent tuning rudimentary AI behavior, such as maneuvering in the game environment [42].  
Yet another explanation may be the tradeoff between authorial control and autonomy. While it can be said that some games and even game genres (e.g., RTS games) thrive on autonomy (e.g., to deliver challenging opponents) it is also true that others purposely minimize autonomy in order to deliver a specific experience to their players (e.g., World of Warcraft) [141]. Furthermore, even when autonomy is desirable, if the AI agent is not properly constrained it can exhibit unpredictable (also referred to as emergent) or inappropriate behavior not anticipated by the game designers/programmers. Such unanticipated behavior in turn may ruin the whole gaming experience. Since AI agents based on NNs, for example, are black boxes for the most part, debugging and preventing such behavior may prove to be a hard task [142]. Of course, these issues do not apply equally to all learning techniques  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Research Topics** & **Learning Techniques** \\ \hline DDA & SL/UL [23; 24], DL [140], EA [25] \\ GCR & SL/UL [27], DL [69] \\ GVGP & SL/UL [53] \\ MaM/MiM & PGM [124; 125], CBR [129; 130], DL [28], RL [107; 108] \\ OM & PGM [121; 122; 123], CBR [131], SL/UL [29; 56; 57], EA [31] \\ PM & SL/UL [24; 32; 33; 34; 58; 59], EA [35; 84] \\ LfD & PGM [43; 124], SL/UL [37; 54; 55], DL [38; 65], EA [82], RL [101; 102; 103; 104; 105] \\ TL & CBR [39], DL [67] \\ PCG & EA [41; 85; 86; 87; 88; 89; 90], RL [106] \\ GDST & PGM [43], EA [44; 45] \\ AGAI & EA [5; 6; 46; 47; 49], RL [8; 48; 50], DS [132; 133; 134] \\ \hline \hline \end{tabular}
\end{table}
Table 2: Table presenting some of the most common game AI research topics together with examples of proposed implementations using the learning techniques discussed. To ease the readability of the table the acronyms of the research topics are repeated here. Dynamic Difficulty Adjustment (DDA), Game Content Recommendation (GCR), General Video Game Playing (GVGP), macromanagement (MaM), micromanagement (MiM), Opponent Modeling (OM), Player Modeling (PM), Learning from Demonstration (LfD), Transfer Learning (TL), Procedural Content Generation (PCG), Game Design Support Tools (GDST) and Adaptive Game AI (AGAI).  
(e.g., DS generates human-readable scripts that can be manually verified). Overall (and attending to the reasons above), implementing learning in games, whether offline or online, increases the difficulty of implementing and testing the game.  
Despite this, learning techniques (NNs in this case) have been used in commercial games such as Black & White from Lionhead Studios (where the objective of the player is to teach and train his own creature) [143] and Supreme Commander 2 (to control the fight or flight behavior of AI controlled platoons) [142] with some degree of success. The use of learning techniques is however not confined to the task of modeling the behaviors of the agents. In fact, these techniques have also been used in the video game industry to help improve the gaming experience of commercial video games via data analytics. Examples of this include the use of ML techniques to cluster players in Battlefield: Bad Company 2 using game telemetry and to create models of the players from gameplay traces both in terms of their performance in World of Warcraft [141] and playing behavior in Tomb Raider: Underworld [33].