Title: Adversarial Policy Transfer in Mixed Cooperative-Competitive Games
Primary area: reinforcement learning
Keywords: Multi-Agent Reinforcement Learning, Transfer Learning, Trustworthy AI

Abstract: Equilibrium learning in mixed cooperative–competitive games remains a central challenge. Adversarial policies expose vulnerabilities in victim agents through task-irrelevant actions, a problem well studied in two-agent zero-sum games and only recently extended to MARL. Existing approaches often overfit to specific scenarios and lack generalization, requiring millions of interactions to adapt to new settings, which is impractical
in real world. By contrast, transferable MARL methods can generalize across tasks but focus on overpowering opponents rather than strategically exploiting their weaknesses.
Here we propose a transferable adversarial policy framework for mixed cooperative–competitive games that enables zero-shot attacks in previously unseen scenarios. Our approach has two key components. First, \emph{adversarial tactic acquisition} iteratively extracts attack strategies that reliably deceive victim agents, using large language models (LLMs) during training and Bayesian inference to weight tactics at test time. Second, \emph{adversarial scene decomposition} partitions attack scenarios into smaller, transferable subgames that consistently elicit adversarial behaviour, based on the interactions between attacker and victim teams. 
Empirically, we demonstrate adversarial policy transfer in \emph{StarCraft II} and \emph{MAgent} across 20 tasks with up to 64 victim agents, varying in number, type and policy. Training against our attack addresses common vulnerabilities in victim policies and enhances robustness to subsequent re-attacks.

TL;DR: We propose a transferable adversarial policy framework for mixed cooperative-competitive games.