System Message Generation for User Preferences using Open-Source Models
Authors: name: Minbyul Jeong, name: Jungho Cho, name: Minsoo Khang, name: Dawoon Jung, name: Teakgyu Hong

System messages play a crucial role in interactions with large language
models (LLMs), often serving as prompts to initiate conversations. Through
system messages, users can assign specific roles, perform intended tasks,
incorporate background information, specify various output formats and
communication styles. Despite such versatility, publicly available data are
often lack system messages and subject to strict license constraints in the
industry field. Manual labeling of publicly available data with system messages
that align with user instructions demands significant resources. In view of
such challenges, our work introduces SysGen, a pipeline for generating system
messages with better aligned assistant responses from the supervised
fine-tuning dataset without system messages. Training on SysGen data has
demonstrated substantial improvements in the alignment of model responses with
system messages and user instructions, as demonstrated across various
open-source models on the Multifacet benchmark, while maintaining minimal
impact on other unseen benchmarks such as Open LLM Leaderboard 2. Our
qualitative analysis highlights the importance of diverse system messages to
ensure better adaptability across different contexts.