Structured Reward Shaping using Signal Temporal Logic specifications Deep reinforcement learning has become a popular technique to train autonomous agents to learn control policies that enable them to accomplish complex tasks in uncertain environments. A key component of an RL algorithm is the definition of a reward function that maps each state and an action that can be taken in that state to some realvalued reward. Typically, reward functions informally capture an implicit (albeit vague) specification on the desired behavior of the agent. In this paper, we propose the use of the logical formalism of Signal Temporal Logic(STL) as a formal specification for the desired behaviors of the agent. Furthermore, we propose algorithms to locally shape rewards in each state with the goal of satisfying the highlevel STL specification. We demonstrate our technique on two case studies, a cartpole balancing problem with a discrete action space, and controlling the actuation of a simulated quadrotor for pointtopoint movement.The proposed framework is agnostic to any specific RL algorithm, as locally shaped rewards can be easily used in concert with any deep RL algorithm.

Tactical Reward Shaping: Bypassing Reinforcement Learning with StrategyBased Goals Deep Reinforcement Learning (DRL) has shown its promising capabilities to learn optimal policies directly from trial and error. However, learning can be hindered if the goal of the learning, defined by the reward function, is not optimal. We demonstrate that by setting the goaltarget of competition in a counterintuitive but intelligent way, instead of heuristically trying solutions through many hours, the DRL simulation can quickly converge into a winning strategy. The ICRADJI RoboMaster AI Challenge is a game of cooperation and competition between robots in a partially observable environment, quite similar to the CounterStrike game. Unlike the traditional approach to games, where the reward is given at winning the match or hitting the enemy, our DRL algorithm rewards our robots when in a geometricstrategic advantage, which implicitly increases the winning chances. Furthermore, we use Deep Q Learning (DQL) to generate multiagent paths for moving, which improves the cooperation between two robots by avoiding collision. Finally, we implement a variant A* algorithm with the same implicit geometric goal as DQL and compare results. We conclude that a wellset goal can put in question the need for learning algorithms, with geometricbased searches outperforming DQL in many orders of magnitude.