Reinforcement learning (RL) has advanced greatly in the past few years with the employment of effective deep neural networks (DNNs) on the policy networks [16, 23] . With the great effectiveness came serious vulnerability issues with DNNs that small adversarial perturbations on the input can change the output of the network [34] . Several works have pointed out that learned agents with a DNN policy network can be manipulated against achieving the original task through a sequence of small perturbations on the input states [10] . In this paper, we demonstrate furthermore that it is also possible to impose an arbitrary adversarial reward on the victim policy network through a sequence of attacks. Our method involves the latest adversarial attack technique, Adversarial Transformer Network (ATN) [1] , that learns to generate the attack and is easy to integrate into the policy network. As a result of our attack, the victim agent is misguided to optimise for the adversarial reward over time. Our results expose serious security threats for RL applications in safety-critical systems including drones [8] , medical analysis [26] , and self-driving cars [31] .
Tretschk et al. REF also aimed to compute a sequence of perturbations, generated by a learned feed-forward DNN, such that the perturbed states misguide the victim policy to follow an arbitrary adversarial reward over time.
Sequential Attacks on Agents for Long-Term Adversarial Goals
ArXiv
ArXiv
Computer Science Mathematics