Before revision:  Policies learned through self-play may adopt arbitrary conventions and  rely on multi-step counterfactual reasoning based on  assumptions about other agents' actions and thus fail when paired with humans or independently trained agents .
After revision:  Policies learned through self-play may adopt arbitrary conventions and rely on multi-step reasoning based on assumptions about other agents' actions and thus fail when paired with humans or independently trained agents .