LLMtimesMapReduce: Simplified Long-Sequence Processing using Large
  Language Models
Authors: name: Zihan Zhou; name: Chong Li; name: Xinyi Chen; name: Shuo Wang; name: Yu Chao; name: Zhili Li; name: Haoyu Wang; name: Rongqiao An; name: Qi Shi; name: Zhixing Tan; name: Xu Han; name: Xiaodong Shi; name: Zhiyuan Liu; name: Maosong Sun
Published: 2024-10-12 03:13:44

Abstract: Enlarging the context window of large language models (LLMs) has become a
crucial research area, particularly for applications involving extremely long
texts. In this work, we propose a novel training-free framework for processing
long texts, utilizing a divide-and-conquer strategy to achieve comprehensive
document understanding. The proposed LLMtimesMapReduce framework splits the
entire document into several chunks for LLMs to read and then aggregates the
intermediate answers to produce the final output. The main challenge for
divide-and-conquer long text processing frameworks lies in the risk of losing
essential long-range information when splitting the document, which can lead
the model to produce incomplete or incorrect answers based on the segmented
texts. Disrupted long-range information can be classified into two categories:
inter-chunk dependency and inter-chunk conflict. We design a structured
information protocol to better cope with inter-chunk dependency and an
in-context confidence calibration mechanism to resolve inter-chunk conflicts.
Experimental results demonstrate that LLMtimesMapReduce can outperform
representative open-source and commercial long-context LLMs, and is applicable
to several different models.