QRM e M : Unleash the Length Limitation through Question then Reflection Memory Mechanism (2024)

Abstract: While LLMs have made notable advancements in natural language processing, they continue to struggle with processing extensive text. Memory mechanisms offer a flexible solution for managing long contexts, utilizing techniques such as compression, summarization, and structuring to facilitate nuanced and efficient handling of large volumes of text. However, existing techniques face challenges with static knowledge integration, leading to insufficient adaptation to task-specific needs and missing multi-segmentation relationships, which hinders the dynamic reorganization and logical combination of relevant segments during the response process. To address these issues, we introduce a novel strategy, Question then Reflection Memory Mechanism (QRMeM), which incorporates a dual-structured memory pool. This pool synergizes static textual content with structured graph guidance, fostering a reflective trial-and-error approach for navigating and identifying relevant segments. Our evaluation across multiple-choice questions (MCQ) and multi-document question answering (Multi-doc QA) benchmarks showcases QRMeM’s enhanced performance compared to existing approaches.

## introduction large language models (llms) have achieved significant advancements in natural language processing. nevertheless, they encounter difficulties handling extensive information, underscoring the necessity for efficient long-context processing. expanding the context window size is a crucial way to handle the problem, including positional embedding and key-value cache @xcite @xcite . however, the inclusion of more information inevitably results in the accumulation of irrelevant document content. ## methodology the components of qrmem are divided into dualstructure memory construction and reflectionbased relevant segments navigation. ## conclusion in this study, we propose the question-then-reflection memory mechanism, which enables the utilization of structured knowledge to guide the mining of supporting segments. qrmem aligns with the query's needs by using structured memory for guidance, and original segments are included as long-term static memory to avoid information bias. we introduce entity trial, graph ex-pansion search, and qrmem as methodologies to navigate through structured memory and find relevant segments. qrmem reflects on errors from past experiences during multiple iterations, allowing the model to learn from its mistakes and improve performance. experimental results on mcq and multi-doc qa demonstrate that our model achieves superior results with various methods, validating the effectiveness of our approach.