Learning Optimal Parameterized Policy for High Level Strategies in a Game Setting Complex and interactive robot manipulation skills such as playing a game of table tennis against a human opponent is a multifaceted challenge and a novel problem. Accurate dynamic trajectory generation in such dynamic situations and an appropriate controller in order to respond to the incoming table tennis ball from the opponent is only a prerequisite to win the game. Decision making is a major part of an intelligent robot and a policy is needed to choose and execute the action which receives highest reward. In this paper, we address this very important problem on how to learn the higher level optimal strategies that enable competitive behaviour with humans in such an interactive game setting. This paper presents a novel technique to learn a higher level strategy for the game of table tennis using PQ Learning (a mixture of Pavlovian learning and Qlearning) to learn a parameterized policy. The cooperative learning framework of Kohenon Self Organizing Map (KSOM) along with Replay Memory is employed for faster strategy learning in this short horizon problem. The strategy is learnt in simulation, using a simulated human opponent and an ideal robot that can perform hitting motion in its workspace accurately. We show that our method is able to improve the average received reward significantly in comparison to the other stateoftheart methods.

Tactical Reward Shaping: Bypassing Reinforcement Learning with StrategyBased Goals Deep Reinforcement Learning (DRL) has shown its promising capabilities to learn optimal policies directly from trial and error. However, learning can be hindered if the goal of the learning, defined by the reward function, is not optimal. We demonstrate that by setting the goaltarget of competition in a counterintuitive but intelligent way, instead of heuristically trying solutions through many hours, the DRL simulation can quickly converge into a winning strategy. The ICRADJI RoboMaster AI Challenge is a game of cooperation and competition between robots in a partially observable environment, quite similar to the CounterStrike game. Unlike the traditional approach to games, where the reward is given at winning the match or hitting the enemy, our DRL algorithm rewards our robots when in a geometricstrategic advantage, which implicitly increases the winning chances. Furthermore, we use Deep Q Learning (DQL) to generate multiagent paths for moving, which improves the cooperation between two robots by avoiding collision. Finally, we implement a variant A* algorithm with the same implicit geometric goal as DQL and compare results. We conclude that a wellset goal can put in question the need for learning algorithms, with geometricbased searches outperforming DQL in many orders of magnitude.