  Growing at a very fast pace, modern autonomous systems will soon be deployed
at scale, opening up the possibility for cooperative multi-agent systems. By
sharing information and distributing workloads, autonomous agents can better
perform their tasks and enjoy improved computation efficiency. However, such
advantages rely heavily on communication channels which have been shown to be
vulnerable to security breaches. Thus, communication can be compromised to
execute adversarial attacks on deep learning models which are widely employed
in modern systems. In this paper, we explore such adversarial attacks in a
novel multi-agent setting where agents communicate by sharing learned
intermediate representations. We observe that an indistinguishable adversarial
message can severely degrade performance, but becomes weaker as the number of
benign agents increase. Furthermore, we show that transfer attacks are more
difficult in this setting when compared to directly perturbing the inputs, as
it is necessary to align the distribution of communication messages with domain
adaptation. Finally, we show that low-budget online attacks can be achieved by
exploiting the temporal consistency of streaming sensory inputs.
