Title: Enhancing Conversational Agents with Skill-of-Mind-Infused Large Language Model
Primary area: datasets and benchmarks
Keywords: Skill-of-Mind, Large Language Model, Social Interaction, Conversational Skill

Abstract: To foster social bonding, humans naturally develop the ability to select appropriate conversational skills (e.g., empathy) based on situational context—a cognitive process we term skill-of-mind. However, LLMs often struggle to generate human-like responses in complex social dialogues. To address this, we propose a 100K skill-of-mind-annotated conversation dataset, Multifaceted Skill-of-Mind, which includes 38 conversational skills across various interactive scenarios (e.g., chitchat), grounded in diverse social contexts (e.g., demographics). Using this dataset, we introduce a new family of skill-of-mind-infused LLMs, Thanos, with model sizes of 1B, 3B, and 8B parameters. We also introduce a comprehensive benchmark suit, ThanosBench, for assessing both capabilities of skill-of-mind and response generation in LLMs. Through extensive experiments evaluating 12 LLMs, Thanos demonstrates performance comparable to Claude-3.5-Sonnet, even outperforming LLaMA-3.1-405B. Specifically, Thanos enhances LLM-generated responses, making them more human-favorable and empathetic communication. Because we find out that recent high-performing LLMs still struggle to exhibit superior skill-of-mind capabilities, we believe it is invaluable to highlight the inherent challenges in this area.

TL;DR: We introduce a new skill-of-mind-infused LLM families with model sizes of 1B, 3B, and 8B and demonstrate its effectiveness on ThanosBench.