Before revision:  Policies learned through self-play may adopt arbitrary conventions and  rely on multi-step counterfactual reasoning based on  assumptions about other agents' actions and thus fail when paired with humans or independently trained agents . In contrast, no current methods can learn optimal policies that are fully grounded, i.e., do not rely on counterfactual information from observing other agents' actions.
After revision:  Policies learned through self-play may adopt arbitrary conventions and rely on multi-step counterfactual reasoning based on assumptions about other agents' actions and thus fail when paired with humans or independently trained agents at test time.