SOTOPIA-π: Interactive Learning of Socially Intelligent Language
  Agents
Authors: name: Ruiyi Wang; name: Haofei Yu; name: Wenxin Zhang; name: Zhengyang Qi; name: Maarten Sap; name: Graham Neubig; name: Yonatan Bisk; name: Hao Zhu
Published: 2024-03-13 17:17:48

Abstract: Humans learn social skills through both imitation and social interaction.
This social learning process is largely understudied by existing research on
building language agents. Motivated by this gap, we propose an interactive
learning method, SOTOPIA-pi, improving the social intelligence of language
agents. This method leverages behavior cloning and self-reinforcement training
on filtered social interaction data according to large language model (LLM)
ratings. We show that our training method allows a 7B LLM to reach the social
goal completion ability of an expert model (GPT-4-based agent), while improving
the safety of language agents and maintaining general QA ability on the MMLU
benchmark. We also find that this training paradigm uncovers some difficulties
in LLM-based evaluation of social intelligence: LLM-based evaluators
overestimate the abilities of the language agents trained specifically for
social interaction.