Multi-LLM Text Summarization
Authors: name: Jiangnan Fang; name: Cheng-Tse Liu; name: Jieun Kim; name: Yash Bhedaru; name: Ethan Liu; name: Nikhil Singh; name: Nedim Lipka; name: Puneet Mathur; name: Nesreen K. Ahmed; name: Franck Dernoncourt; name: Ryan A. Rossi; name: Hanieh Deilamsalehy
Published: 2024-12-20 01:55:26

Abstract: In this work, we propose a Multi-LLM summarization framework, and investigate
two different multi-LLM strategies including centralized and decentralized. Our
multi-LLM summarization framework has two fundamentally important steps at each
round of conversation: generation and evaluation. These steps are different
depending on whether our multi-LLM decentralized summarization is used or
centralized. In both our multi-LLM decentralized and centralized strategies, we
have k different LLMs that generate diverse summaries of the text. However,
during evaluation, our multi-LLM centralized summarization approach leverages a
single LLM to evaluate the summaries and select the best one whereas k LLMs are
used for decentralized multi-LLM summarization. Overall, we find that our
multi-LLM summarization approaches significantly outperform the baselines that
leverage only a single LLM by up to 3x. These results indicate the
effectiveness of multi-LLM approaches for summarization.