ChatGPT-steered Editing Instructor for Customization of Abstractive
  Summarization
Authors: name: Wen Xiao; name: Yujia Xie; name: Giuseppe Carenini; name: Pengcheng He
Published: 2023-05-04 01:12:35

Abstract: Tailoring outputs of large language models, such as ChatGPT, to specific user
needs remains a challenge despite their impressive generation quality. In this
paper, we propose a tri-agent generation pipeline consisting of a generator, an
instructor, and an editor to enhance the customization of generated outputs.
The generator produces an initial output, the user-specific instructor
generates editing instructions, and the editor generates a revised output
aligned with user preferences. The inference-only large language model
(ChatGPT) serves as both the generator and the editor, while a smaller model
acts as the user-specific instructor to guide the generation process toward
user needs. The instructor is trained using editor-steered reinforcement
learning, leveraging feedback from the large-scale editor model to optimize
instruction generation. Experimental results on two abstractive summarization
datasets demonstrate the effectiveness of our approach in generating outputs
that better fulfill user expectations.