# Hybrid Training Strategies: Improving Performance of Temporal Difference Learning in Board Games +
## 5 Conclusions  
Several training strategies for a temporal difference learning agent, implemented as simulated opponent players with different rules, have been presented, described, created, and assessed in this study.  
We have proven that if the right training method is used and a certain degree of hybridness is included in the model, the TD algorithm's convergence may be seriously enhanced. When the Attack-Defense strategy is used, perfection is reached in the classic tic-tac-toe board game with fewer than 100,000 games of training. Previously, many millions of games were required to achieve comparable ranks in the case of obtaining convergence, according to earlier research works.  
Hybrid training strategies improve the efficiency of TD learning, reducing the resources needed to be successful (execution time, memory capacity) and ultimately being more cost-effective and power-aware.  
Despite being particularly easy to construct and having a low computing cost, the introduced Attack-Defense training approach (try to win; if not feasible, try not to lose; else make a random move) has shown to be extremely successful in our testing, exceeding the other choices evaluated (even the Combined option). Furthermore, a TD agent that has been taught using this approach may be deemed a perfect tic-tac-toe player, having never lost a game against both random and expert opponents.  
The Attack-Defense strategy presented in this work has also proved to outperform the rest of the strategies in the 4 \(\times\) 4 Connect-4 board game (four rows and four columns), which has a considerably higher number of states than the tic-tac-toe game (an upper-bound estimation being 200,000 legal states).  
Future research will look into the effectiveness of hybridness and the Attack-Defense training technique in board games with a larger space-state complexity (such as Connect-4 for larger board sizes, checkers, or go).  
Two of the most popular applications of reinforcement learning are game playing and control problems in robotics. The underlying concepts behind the training technique presented in this research work (confront the agent with an opponent with partial knowledge of how to win and not to lose, including a certain degree of randomness) could be generalized and applied to training robots in diverse situations, using different environments and degrees of freedom.  
Conceptualization, J.M.C., P.C.-J. and J.F.-C.; funding acquisition, J.M.C.; investigation, J.M.C., P.C.-J. and J.F.-C.; methodology, J.F.-C.; project administration, J.M.C.; resources, J.M.C.; software, J.F.-C.; supervision, J.M.C.; validation, J.F.-C.; visualization, P.C.-J. and J.F.-C.; writing--original draft, J.F.-C.; writing--review and editing, P.C.-J. and J.F.-C. All authors have read and agreed to the published version of the manuscript.  
The research leading to these results has received funding from RoboCity2030-DIH-CM, Madrid Robotics Digital Innovation Hub, S2018/NMT-4331, funded by \"Programas de Actividades I+D en la Comunidad de Madrid\", and cofunded by Structural Funds of the EU.  
Not applicable.  
The data presented in this study are available on request from the corresponding author.  
The authors declare no conflict of interest.