BERT-VBD: Vietnamese Multi-Document Summarization Framework
Authors: name: Tuan-Cuong Vuong, name: Trang Mai Xuan, name: Thien Van Luong

In tackling the challenge of Multi-Document Summarization (MDS), numerous
methods have been proposed, spanning both extractive and abstractive
summarization techniques. However, each approach has its own limitations,
making it less effective to rely solely on either one. An emerging and
promising strategy involves a synergistic fusion of extractive and abstractive
summarization methods. Despite the plethora of studies in this domain, research
on the combined methodology remains scarce, particularly in the context of
Vietnamese language processing. This paper presents a novel Vietnamese MDS
framework leveraging a two-component pipeline architecture that integrates
extractive and abstractive techniques. The first component employs an
extractive approach to identify key sentences within each document. This is
achieved by a modification of the pre-trained BERT network, which derives
semantically meaningful phrase embeddings using siamese and triplet network
structures. The second component utilizes the VBD-LLaMA2-7B-50b model for
abstractive summarization, ultimately generating the final summary document.
Our proposed framework demonstrates a positive performance, attaining ROUGE-2
scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art
baselines.