# A Survey of Planning and Learning in Games
## 3 Learning in Games
### Discussion  
The previous subsection presented some learning techniques used in the context of game AI and highlighted the different application uses of these techniques in the domain of games. An interesting observation that can be derived from that discussion is the role that learning techniques have played in the context of classical board games, not only as a way of reducing the need for expert domain knowledge and automating some implementation details (e.g., the discovery of features to represent the board positions and the implementation of the evaluation functions used to evaluate them) but also as a successful alternative to search techniques (e.g., Minimax) in games where these techniques suffered from poor performance (e.g., backgammon). In addition, some of these techniques seem to have gained a lot of popularity in solving particular tasks (e.g., EAs in PCG and SL/UL in PM). Another interesting fact is the prolific integration of DL with both RL and EAs. Since the success of the DQN algorithm, DRL research has gained significant popularity and is currently one of the most popular techniques of choice. In fact, over the years multiple simulation frameworks, such as the Arcade Learning Environment (ALE) [135], VizDoom [136], MuJoCo [137], and Unity's ML-Agents Toolkit [138], have been developed with the purpose of providing challenging environments and benchmarks for DRL research. Furthermore, most research dealing, for example, with endowing AI agents with human-like capabilities such as curiosity, inner motivation, dreaming, and imagination is based on DRL techniques. In the case of EAs, neuroevolution is a clear proof of the impact caused by DL in the field. Neuroevolution in turn has pushed the boundaries of DL by devising algorithms that can not only train NNs but also build them (e.g., rtNEAT). Neuroevolution has also proven to be a competitive alternative to DRL as attested by Open AI's work in MuJoCo and Atari [139], further enriching the range of possible technical solutions. Table 1 presents an overall comparison of the different learning techniques presented in terms of some of the most relevant aspects that a researcher/practitioner may (or should) take into account when deciding on which technique to use to solve a specific problem related to game AI (or even other kinds of domain). Regardless of this, all of these techniques have found application in the context of games. Table 2 (found at the end of this subsection) presents a brief summary of some of these applications. Section 5 will present even more examples (including some of the most recent milestones in AI history) of the application of these learning techniques in combination with planning techniques. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Tech.** & **DER** & **RTD** & **ITM** & **AC** & **Popularity** \\ \hline SL/UL & medium/high & yes & low & medium & high \\ DL & low/medium & (1) & low & low/ medium & very high \\ EA & low/medium & no & low & low/ medium & very high \\ RL & low/medium & no & low & low/ medium & very high \\ PGM & high & yes & medium/ high & medium & medium \\ CBR & medium/high & yes & medium/ high & medium & low/medium \\ DS & high & no & high & very high & low/medium \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Table presenting a comparison of the different learning techniques discussed. The different aspects comprising the comparison are scored according to an overall score, as these values may differ according to specific algorithms (or variations thereof), research topics, or even game genres. The acronyms used stand for respectively: Domain Expertise Required (DER), Requires a Training Dataset (RTD), Interpretability of the Trained Model (ITM) and Authorial Control (AC). In the table, (1) is used to denote that the need for a training dataset depends on the training method used to train the NNs.