EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation
Authors: Taeho Hwang, Sukmin Cho, Soyeong Jeong, Hoyun Song, SeungYoon Han, Jong C. Park
Accepted at: acl25

EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation

Taeho Hwang1   Sukmin Cho1   Soyeong Jeong2   Hoyun Song1
SeungYoon Han1   Jong C. Park1*
1School of Computing, 2Graduate School of AI
Korea Advanced Institute of Science and Technology
{doubleyyh,nelllpic,starsuzi,hysong,seungyoonee,jongpark}@kaist.ac.kr

Abstract
We introduce EXIT, an extractive context compression framework that enhances both the effectiveness and efficiency of retrieval-augmented generation (RAG) in question answering (QA). Current RAG systems often struggle when retrieval models fail to rank the most relevant documents, leading to the inclusion of more context at the expense of latency and accuracy. While abstractive compression methods can drastically reduce token counts, their token-by-token generation process significantly increases end-to-end latency. Conversely, existing extractive methods reduce the latency but rely on independent, non-adaptive sentence selection, failing to fully utilize contextual information. EXIT addresses these limitations by classifying sentences from retrieved documents—while preserving their contextual dependencies—enabling parallelizable, context-aware extraction that adapts to query complexity and retrieval quality. Our evaluations on both single-hop and multi-hop QA tasks show that EXIT consistently surpasses existing compression methods and even uncompressed baselines in QA accuracy, while also delivering substantial reductions in inference time and token count. By improving both effectiveness and efficiency, EXIT provides a promising direction for developing scalable, high-quality QA solutions in RAG pipelines. Our code is available at https://github.com/ThisIsHwang/EXIT.

1 Introduction
Retrieval-Augmented Generation (RAG) (Lewis et al., 2020b; Khandelwal et al., 2020) is the task of enhancing the responses of Large Language Models (LLMs) with relevant external contexts or documents. By grounding answers in evidence, RAG systems have gained much attention for mitigating hallucination issues (Ram et al., 2023; Li et al., 2023c) and improving factual reliability (Jeong et al., 2024; Xia et al., 2024b).

Figure 1: Average QA accuracy (EM) and efficiency (Total Latency) across various compression methods using Contriever-MSMARCO as the retriever and Llama-3.1-8b-Instruct as the reader. Experiments were conducted on a single A100-80GB GPU, and latency (in seconds) was averaged over all samples from these datasets.

However, they face significant challenges in practical deployment, as retrieval models sometimes fail to rank the most relevant documents at the top (Robertson and Zaragoza, 2009; Izacard et al., 2022). One potential solution is to retrieve a larger set of documents to ensure the coverage of the necessary information, but this approach compromises both effectiveness and efficiency. Specifically, for effectiveness, LLMs often struggle with processing long contexts, overlooking critical information located in the middle of contexts (Liu et al., 2024). Additionally, irrelevant information in retrieved documents can act as distractors, significantly degrading the overall QA performance (Shi et al., 2023; Li et al., 2023b; Wu et al., 2024). From an efficiency perspective, increasing the context size raises inference latency—due to quadratic complexity in attention computation (Xia et al., 2024a)—and API costs tied to input length1. Moreover, the context window limitations inherent in LLM architectures set strict upper bounds on the maximum input size (Peng et al., 2024).

To address these challenges, context compression has emerged as a promising solution, condensing essential information from multiple retrieved

1https://openai.com/api/pricing/
contexts through either abstractive or extractive approaches. While they can reduce inference time and filter out irrelevant information, both approaches still have notable drawbacks. Specifically, abstractive compression methods—often implemented via autoregressive generation—summarize or rewrite documents into a single condensed passage (Li et al., 2023d; Xu et al., 2024; Yoon et al., 2024; Li et al., 2024; Wang et al., 2023), significantly increasing end-to-end latency due to their token-by-token generation process. For instance, as illustrated in Figure 1, CompAct (Yoon et al., 2024), a representative abstractive approach, takes over 8 seconds to process just five documents for a single query, whereas using the original document without compression takes only 1 second.

On the other hand, extractive compression approaches can offer a more efficient alternative (Xu et al., 2024; Jiang et al., 2023, 2024), by selecting relevant textual segments (e.g., sentences, or even token-level excerpts) directly from retrieved documents. This strategy reduces both compression time and overall latency. However, current extractive methods have yet to reach their full potential in terms of effectiveness (Choi et al., 2021; Pan et al., 2024). They often rely on rigid selection criteria that do not adapt well to variations in query complexity or the quality of retrieved documents, and they frequently neglect to fully leverage the broader context when choosing which tokens or sentences to retain. Specifically, as illustrated in Figure 1, while extractive approaches such as RECOMP-Extr (Xu et al., 2024) achieve minimal compression time, their inability to dynamically adjust selection processes results in suboptimal QA performance.

Therefore, in this work, we propose a novel compression framework for RAG, EXtractIve ContextT compression (EXIT), designed to enhance both effectiveness and efficiency by improving efficiency through an extractive compression and by enhancing effectiveness through dynamic, context-aware sentence selection. Specifically, as shown in Figure 2, EXIT operates in three stages: (1) splitting retrieved documents into sentences, (2) performing parallelizable binary classification (“Yes” or “No”) on each sentence to assess its relevance while considering its full document context, rather than evaluating them independently, and (3) recombining selected sentences while preserving their original order. Therefore, as shown in Figure 1, EXIT frames context compression as a sentence classification problem, enabling it to outperform both compression methods and the uncompressed baseline in terms of speed. Specifically, it reduces processing time from several seconds to about 1 second. Moreover, by leveraging context-aware and adaptive sentence selection, EXIT also surpasses other extractive methods in accuracy. Also, we note that EXIT operates as a plug-and-play module that can be seamlessly integrated into any existing RAG pipeline without architectural modifications.

We evaluate EXIT on both single-hop QA tasks (NQ (Kwiatkowski et al., 2019), TQA (Joshi et al., 2017)) and multi-hop QA tasks (HQA (Yang et al., 2018), 2WikiMultiHopQA (Ho et al., 2020)). Experimental results demonstrate that EXIT not only improves effectiveness over both abstractive and extractive compression baselines but also significantly reduces latency compared to abstractive methods and the uncompressed baseline.

Our contributions are as follows:

• We identify and address the key weaknesses of existing context compression methods: abstractive methods incur prohibitive latency, while traditional extractive methods rely on rigid, non-adaptive content selection.

• We propose EXIT (EXtractIve ContextT Compression), an extractive compression framework that dynamically adjusts to query complexity and retrieval quality.

• We demonstrate, through extensive experiments, that EXIT surpasses previous compression methods and uncompressed retrievals, improving QA performance while significantly reducing both token counts and end-to-end latency.

2 Related Work

Retrieval-Augmented Generation. RAG enhances LLMs by grounding responses in retrieved external documents (Lewis et al., 2020b; Shi et al., 2024; Ram et al., 2023). However, increasing retrieved content often leads to longer contexts, higher inference costs (Xia et al., 2024a), and retrieval noise (Shi et al., 2023; Liu et al., 2024), impacting efficiency and accuracy.

To mitigate these challenges, ad-hoc methods have been proposed at different stages of the retrieval pipeline, including pre-retrieval and post-retrieval methods. Pre-retrieval methods, such as
Figure 2: Overview of our framework. First, the retrieved document is split into sentences. Next, each sentence is classified as either “Yes” or “No” using the Compressor. Finally, sentences with scores above the threshold are recombined in their original order to complete the compression.

Landmark Embedding (Luo et al., 2024) and Late Chunking (Günther et al., 2024), improve document representations to enhance retrieval effectiveness but lack adaptive filtering mechanisms after retrieval and may be difficult to integrate orthogonally with existing retrieval methods. Post-retrieval methods, including reranking (Nogueira and Cho, 2019; Qin et al., 2023; Li et al., 2023a) and context compression (Xu et al., 2024; Yoon et al., 2024; Li et al., 2024; Jiang et al., 2024), refine retrieved documents through reordering or pruning. However, most post-retrieval methods overlook query complexity and operate on a fixed number of retrieved items, limiting their adaptability in balancing effectiveness and efficiency.

We introduce EXIT, a novel post-retrieval compression framework that adaptively filters query-relevant sentences while discarding redundancy. Unlike existing methods, EXIT dynamically adjusts to query complexity and retrieval quality, balancing efficiency and effectiveness. Its lightweight, adaptive design seamlessly integrates into RAG pipelines while remaining orthogonal to pre-retrieval methods.

Context Compression. Context compression is a practical remedy for handling increasingly extensive prompt lengths in RAG pipelines. Existing methods commonly fall into soft or hard compression. Soft compression focuses on embedding prompts into compact continuous representations (Wingate et al., 2022; Mu et al., 2023; Ge et al., 2024; Chevalier et al., 2023; Cheng et al., 2024), but requires extensive training and architectural changes, making it unsuitable for black-box LLMs. Hard compression, by contrast, removes non-essential textual content directly (Li et al., 2023e; Jiang et al., 2023), offering a plug-and-play solution even compatible with API-based models such as ChatGPT (OpenAI, 2023).

Hard compression are further divided into abstractive and extractive methods. Abstractive methods employ autoregressive models to generate query-focused summaries, thus reducing token counts at the cost of additional latency and potential hallucinations (Zhao et al., 2020). For example, RECOMP-Abst (Xu et al., 2024) uses a T5-based summarizer for token reduction but requires dataset-specific training and slows inference. CompAct (Yoon et al., 2024) and Refiner (Li et al., 2024) take this method further by leveraging even larger LLMs with 7B parameters, compounding latency issues and increasing resource demands. Extractive methods, such as RECOMP-Extr (Xu et al., 2024), directly select salient segments (e.g., sentences or tokens) from retrieved documents. This avoids the autoregressive bottleneck and mitigates hallucinations. However, their static and context-agnostic selection of only a few sentences per document limits its performance. Similarly, token-level methods such as LLMLingua family (Li et al., 2023e; Jiang et al., 2023; Pan et al., 2024; Jiang et al., 2024) can distort semantic coherence by removing key entities or splitting essential facts.

In short, abstractive methods offer strong com-
pression but suffer from latency and potential hallucinations, while extractive methods are often rigid and lack context awareness. Our work addresses these limitations by proposing a parallelizable, context-aware extractive compression framework. It adaptively selects sentences at scale, optimizing accuracy and speed in complex retrieval scenarios.

3 Method

In this section, we first present our problem formulation, the RAG pipeline with a compression stage, and our novel compression framework, EXIT, which is designed to extract key evidence for answering in a parallel manner.

3.1 Problem Formulation

RAG Pipeline with Compression. Given a query \( q \) and a document corpus \( \mathcal{C} \), a RAG pipeline first retrieves Top-\( k \) relevant document set \( D \):

\[
D = \{d_1, \ldots, d_k\} = \text{Retriever}(q, \mathcal{C}),
\]

The retrieved documents within the document set \( D \) are then processed by a compression module that preserves query-relevant information while significantly reducing input length:

\[
D' = \text{Compressor}(q, D) \text{ s.t. } l(D') \ll l(D),
\]

where \( l(\cdot) \) represents the function calculating the number of tokens in the document set. After compression, the number of tokens included in \( D' \) is substantially decreased compared to \( D \). Finally, an LLM generates the answer \( a \) using the compressed set \( D' \) and the given query \( q \):

\[
a = \text{LLM}(q, D').
\]

Objectives of Compression. Effective compression in the RAG pipeline must satisfy three key criteria: (1) Token Reduction—\( D' \) should have fewer tokens to speed up answer generation; (2) Retention of Key Evidence—essential information must be preserved to maintain answer accuracy; and (3) Efficient Processing—compression should be fast enough to avoid introducing significant latency.

While token count influences the reader’s reading time, it alone does not capture the full latency of the pipeline. In practice, end-to-end latency also includes the time spent on the compression step itself. Therefore, an efficient RAG system must minimize both the size of the input and the total time taken from retrieval to answer generation.

3.2 Extractive Context Compression (EXIT)

To achieve three objectives of the compression step, EXIT consists of three main components: sentence-level decomposition, context-aware relevance classification, and document reassembly. Importantly, EXIT functions as a plug-and-play module that integrates seamlessly into existing RAG pipelines, independent of the specific retriever or reader model used.

Sentence-Level Decomposition. In Step 1 of Figure 2, EXIT divides each retrieved document into individual sentences using a rule-based sentence tokenizer. For each document \( d_i \in D \), it produces a sentence set \( S_i = \{s_{i1}, s_{i2}, \ldots, s_{im}\} \), where \( s_{ij} \) is the \( j \)-th sentence in document \( i \). Operating at the sentence level avoids the fragmentation of key phrases and preserves entity relationships that token-level compression techniques (Jiang et al., 2023) often disrupt. As a result, the compressed context preserves both syntactic coherence and semantic integrity, ensuring that key information is retained.

Context-Aware Relevance Classification. To efficiently filter sentences in \( D \) that contain key evidence for answering a query, we introduce a relevance evaluation method based on context awareness and single-token prediction. First, incorporating the entire document \( d_i \) as context is essential, as understanding a sentence often requires the broader document context rather than an isolated sentence. This ensures that no relevant information is overlooked and enables more effective compression. Second, to maintain efficiency, we adopt lightweight single-token prediction instead of multi-token generation (Yoon et al., 2024; Li et al., 2024), which can introduce computational overhead. Inspired by Zhong et al. (2022), we leverage probabilistic classification for sentence relevance assessment. Given query \( q \), document \( d_i \), and sentence \( s_{ij} \), the evaluation model predicts relevance using a binary classification with “Yes” and “No” labels:

\[
r_{ij} = \frac{P("Yes"|q, d_i, s_{ij})}{P("Yes"|q, d_i, s_{ij}) + P("No"|q, d_i, s_{ij})}
\]

where \( P(\cdot|\cdot) \) represents the likelihood from the evaluation model. This computation is parallelized across sentences for efficiency.

EXIT then selects sentences with a relevance score above a predefined threshold \( \tau \), ensuring that only key information is retained. Unlike fixed-size selection, our framework produces an adaptive
number of sentences in the compressed set \( D' \), aligning with prior work (Jeong et al., 2024) that acknowledges query-dependent information needs. This approach optimizes compression while preserving essential evidence.

Document Reassembly. As shown in Step 3 of Figure 2, EXIT reconstructs the compressed document \( D' \) by concatenating selected sentences in their original order, preserving coherence and logical flow (Hwang et al., 2024) for accurate downstream reasoning.

3.3 Classifier Model Training

Training Strategy. Our goal is to train a relevance classifier capable of accurately identifying which sentences provide the evidence required to answer a query. To approximate real-world complexity, we utilize a question-answering dataset that requires multi-sentence reasoning and offers explicit sentence-level annotations of essential information. Leveraging these annotations, we model three typical retrieval outcomes: (1) sentences containing necessary evidence, (2) seemingly relevant sentences missing key details, and (3) entirely irrelevant sentences.

Data Sampling. To train a robust relevance classifier, we construct a diverse dataset reflecting various post-retrieval conditions. Positive samples include sentences essential for correct answers, while hard negatives consist of remaining sentences from the same documents. Additionally, random negatives pair queries with unrelated sentences, helping the model distinguish relevance from noise. This balanced sampling enables effective filtering of non-essential information without relying on explicit supervision.

Training Procedure. Each training instance is represented as \((q, s, d, l)\), where \( q \) is the query, \( s \) is a candidate sentence, \( d \) is the document containing \( s \), and \( l \in \{“Yes”, “No”\} \) indicates whether \( s \) provides the required evidence. We employ a binary cross-entropy loss function to train the classifier:
\[
\mathcal{L} = -1_{l=“Yes”} \log P(“Yes”) - (1-1_{l=“Yes”}) \log P(“No”).
\]
By exposing the classifier to a balanced and diverse set of retrieval scenarios, we improve its ability to generalize and reliably identify sentences that contain the critical evidence for answering queries.

4 Experiment Setups

We conduct comprehensive experiments to evaluate EXIT’s effectiveness and efficiency in context compression for RAG systems. More implementation details of our experiments are in Appendix B.

Datasets. We evaluate on both single-hop and multi-hop question answering datasets: NaturalQuestions (NQ) (Kwiatkowski et al., 2019) and TriviaQA (TQA) (Joshi et al., 2017) for single-hop QA; HotpotQA (HQA) (Yang et al., 2018) and 2WikiMultihopQA (2WIKI) (Ho et al., 2020) for multi-hop QA. We use the test set for TQA evaluations and development sets for all other datasets. For the train dataset for the classifier, we exploit the train split of HQA, which has relevant annotations to each sentence in the multiple documents for a query.

Model Configuration. Our system consists of three primary components. The retriever employs Contriever-MSMARCO (Izacard et al., 2022), a dense retriever fine-tuned on MSMARCO (Nguyen et al., 2016). The EXIT compressor utilizes Gemma-2B-it (Mesnard et al., 2024), optimized for efficient parallel processing. For the reader model, we exploit two scales of instruction-tuned models: Llama3.1-{8, 70}B-Instruct (Dubey et al., 2024).

Baselines. We compare EXIT against the following context compression methods: 1) Original Documents serves as uncompressed baseline. For abstractive methods, 2) RECOMP-Abs (Xu et al., 2024) uses T5-based (775M) summarization tuned for NQ, TQA, HQA (HQA model for 2WIKI), 3) CompAct (Yoon et al., 2024) implements Mistral-7B-based iterative compression with 5-segment blocks, and 4) Refiner (Li et al., 2024) uses Llama2-7B-based compression. For extractive methods, 5) RECOMP-Extr (Xu et al., 2024) employs Contriever-based (110M) sentence-level extraction tuned for NQ, TQA, HQA (HQA model for 2WIKI), and 6) LongLLMLingua (Jiang et al., 2024) uses Llama2-7B-chat for token-level extraction with 0.4 dynamic compression rate.

Evaluation Metrics. We use three metrics: Exact Match (EM) and F1 score to measure effectiveness in question answering, and end-to-end inference latency (Lat.) in seconds to assess efficiency. End-to-end latency captures the total wall-clock time required to generate an answer for a single query, including both compression time and LLM inference time. We report the average latency per sample, computed over the entire evaluation set. This provides a more holistic efficiency metric than token count alone, as it directly reflects the practical runtime performance of the full RAG pipeline.
Table 1: Performance across models and datasets, measured by EM, F1, and inference latency (Lat.). 8B reader experiments were conducted on a single A100-80GB GPU; while 70B reader experiments utilized 4 A100-80GB GPUs in parallel. Best results for each dataset are highlighted in bold, and second best results are underlined. The “Type” column denotes whether a given compressor is abstractive (Abs.) or extractive (Ext.).

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="3">NQ</th>
    <th colspan="3">TQA</th>
    <th colspan="3">HQA</th>
    <th colspan="3">2WIKI</th>
    <th colspan="3">AVG.</th>
  </tr>
  <tr>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
  </tr>
  <tr>
    <td colspan="18"><b>Llama3.1-8B-Instruct</b></td>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>34.6</td><td>47.1</td><td>1.0</td><td>58.8</td><td>68.6</td><td>0.9</td><td>28.1</td><td>38.6</td><td>1.0</td><td>16.1</td><td>24.9</td><td>1.1</td><td>34.4</td><td>44.8</td><td>1.0</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>31.3</td><td>43.2</td><td>1.6</td><td>55.9</td><td>65.7</td><td>1.4</td><td>26.5</td><td>37.0</td><td>2.2</td><td>22.7</td><td>29.1</td><td>2.1</td><td>34.1</td><td>43.7</td><td>1.8</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>32.9</td><td>44.6</td><td>8.5</td><td>58.1</td><td>67.7</td><td>8.8</td><td>28.8</td><td>39.8</td><td>8.3</td><td>16.8</td><td>26.0</td><td>8.1</td><td>34.2</td><td>44.5</td><td>8.4</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>32.9</td><td>45.0</td><td>28.1</td><td>59.2</td><td>68.9</td><td>10.9</td><td>28.8</td><td>40.0</td><td>6.9</td><td>16.8</td><td>25.4</td><td>6.4</td><td>34.4</td><td>44.8</td><td>13.1</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>34.6</td><td>44.6</td><td>0.5</td><td>56.5</td><td>65.1</td><td>0.4</td><td>23.4</td><td>32.8</td><td>0.4</td><td>11.2</td><td>19.6</td><td>0.6</td><td>31.4</td><td>40.5</td><td>0.5</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>30.2</td><td>41.5</td><td>0.9</td><td>59.4</td><td>68.0</td><td>0.8</td><td>28.0</td><td>38.0</td><td>0.8</td><td>21.5</td><td>27.4</td><td>0.9</td><td>34.8</td><td>43.7</td><td>0.9</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td><td>Ext.</td><td><b>35.9</b></td><td><b>47.8</b></td><td><u>0.8</u></td><td><b>60.8</b></td><td><b>69.9</b></td><td><u>0.7</u></td><td><b>30.6</b></td><td><b>41.5</b></td><td><u>0.8</u></td><td><b>24.2</b></td><td><b>30.8</b></td><td><u>0.9</u></td><td><b>37.9</b></td><td><b>47.5</b></td><td><u>0.8</u></td>
  </tr>
  <tr>
    <td colspan="18"><b>Llama3.1-70B-Instruct</b></td>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>35.6</td><td>48.0</td><td>8.6</td><td>65.1</td><td>73.9</td><td>7.7</td><td>33.7</td><td>44.5</td><td>8.3</td><td>20.8</td><td>28.3</td><td>9.1</td><td>38.8</td><td>48.7</td><td>8.4</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>34.1</td><td>47.0</td><td>4.5</td><td>61.3</td><td>70.6</td><td>3.3</td><td>30.3</td><td>40.8</td><td>4.4</td><td>24.2</td><td>30.3</td><td>4.2</td><td>37.5</td><td>47.2</td><td>4.1</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>34.1</td><td>45.4</td><td>11.9</td><td>62.6</td><td>71.1</td><td>11.7</td><td>33.8</td><td>44.1</td><td>11.0</td><td>20.5</td><td>27.4</td><td>11.6</td><td>37.8</td><td>47.0</td><td>11.5</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>35.3</td><td>47.1</td><td>42.5</td><td>64.3</td><td>73.0</td><td>18.3</td><td>33.8</td><td>44.7</td><td>14.6</td><td>21.2</td><td>28.0</td><td>11.2</td><td>38.7</td><td>48.2</td><td>21.6</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>35.8</td><td>45.3</td><td>2.5</td><td>63.5</td><td>71.0</td><td>2.2</td><td>27.6</td><td>36.7</td><td>2.9</td><td>13.8</td><td>19.3</td><td>3.3</td><td>35.2</td><td>43.1</td><td>2.7</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>32.2</td><td>44.0</td><td>4.4</td><td>66.7</td><td>75.2</td><td>3.9</td><td>34.1</td><td>45.3</td><td>4.0</td><td>28.3</td><td><b>34.8</b></td><td>4.3</td><td>40.3</td><td>49.8</td><td>4.1</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td><td>Ext.</td><td><b>36.9</b></td><td><b>49.4</b></td><td><u>3.9</u></td><td><b>67.3</b></td><td><b>75.9</b></td><td><u>3.1</u></td><td><b>37.0</b></td><td><b>48.3</b></td><td><u>3.3</u></td><td><b>28.6</b></td><td><b>34.5</b></td><td><u>3.5</u></td><td><b>42.5</b></td><td><b>52.0</b></td><td><u>3.5</u></td>
  </tr>
</table>

Implementation Details. We perform document retrieval using the December 2018 Wikipedia dump and apply SpaCy for sentence segmentation. The LLM outputs are generated with a temperature of 0.0 and a top-P of 1.0. We set the relevance threshold \( \tau \) to 0.5 based on empirical tuning. All experiments are conducted on A100-SXM4-80GB GPUs.

5 Main Results

Table 1 summarizes our evaluation results across multiple datasets and compression methods. With the 8B reader, EXIT demonstrates strong generalization: although trained solely on HQA, it effectively addresses both single-hop (NQ, TQA) and multi-hop (2WIKI) queries under out-of-domain conditions. Compared to all baseline methods, EXIT consistently improves EM scores—for instance, by 1.3 and 2.0 points on NQ and TQA, and by even larger margins of 2.5 and 8.1 points on HQA and 2WIKI, respectively. Notably, these accuracy gains come with an average latency of just 0.8s, substantially faster than abstractive compression methods.

The benefits of EXIT become more pronounced at larger scales. Using the 70B reader, EXIT surpasses the accuracy of all competing methods, averaging a 3.7-point improvement in EM and a 3.3-point improvement in F1 over the uncompressed baseline. On HQA, it achieves a 3.3-point EM gain while maintaining an efficient 3.5s latency—faster than using uncompressed documents and still competitive with the previously fastest method, RECOMP-Extr, but with significantly higher accuracy. EXIT’s effectiveness and efficiency, especially with larger models, makes it a practical solution for large-scale QA applications.

6 Analyses

We conduct a series of analyses examining EXIT’s robustness, classification performance, latency factors, and design choices under various configurations. Additional experimental results and analyses are provided in Appendix C.

6.1 Robustness Analysis

To examine EXIT’s robustness as the retrieval set size grows, we gradually increased the number of retrieved documents (\( k \in \{1, 5, 10, 20, 30\} \)) with an 8B reader, as shown in Figure 3. We found that EXIT steadily improves EM scores—from 28.2 points at \( k = 1 \) to 33.1 points at \( k = 30 \)—while avoiding the performance degradation seen in RECOMP variants and Refiner at high \( k \) values. Also, we measured the impact on the efficiency of the RAG pipeline with token counts and end-to-end latency, confirming that EXIT significantly reduces context from 4,497.1 tokens to 594.4 tokens (86.8% fewer) at \( k = 30 \), even improving the quality. Also, EXIT’s latency scales nearly linearly (0.48s to 2.71s) and is much faster than the abstraction methods and the uncompressed baseline. These results demonstrate that EXIT consistently delivers
Figure 3: Performance analysis on HQA across different Top-k values (1, 5, 10, 20, 30), comparing accuracy, token retention, and inference latency between baselines and our method. All experiments were conducted on a single A100-80GB GPU.

Figure 4: Comparison of compression and reading latency across baselines and our method in QA setting. Experiments were conducted on a single A100 GPU.

significant accuracy improvements with minimal inference costs, regardless of the number of documents, making it well-suited for tasks involving larger retrieval sets.

6.2 Understanding End-to-End Latency Factors

While previous work has primarily focused on minimizing token counts to reduce reading time, we argue that end-to-end latency—including both compression and generation time—offers a more practical measure of efficiency for RAG pipelines.

Figure 4 presents a breakdown of total latency into compression time and reading time, along with the average token count of the compressed context. Methods such as RECOMP-Abst (6.2% token ratio) and CompAct (10.6%) achieve strong compression but suffer from long compression times (1.46s and 7.99s, respectively), resulting in overall latency that exceeds the uncompressed baseline. By contrast, EXIT achieves a balanced 31.2% token ratio and fast compression (0.36s), reducing total latency to 0.79s—a 23.3% improvement over the uncompressed baseline (1.03s).

We acknowledge that EXIT may retain more content when the question is simple or the retrieved documents are of high quality. However, our goal is not to maximize compression alone, but to minimize total latency while preserving answer accuracy. Aggressive compression in such cases could introduce unnecessary overhead or risk omitting useful context. EXIT’s sentence-level classification balances this trade-off, allowing rapid, context-aware compression with consistent gains in both speed and accuracy, as shown in Table 1.

This analysis highlights that efficient compression is not solely about reducing tokens—it is about achieving the best trade-off between compression time and model input size. EXIT is designed with this principle in mind, making it a practical solution for real-world RAG applications.

6.3 Computational Complexity and Efficiency Trade-offs

To strengthen the justification of EXIT’s efficiency, we analyze not only latency but also computational complexity, measured in TFLOPs, using DeepSpeed’s FlopsProfiler (Rasley et al., 2020). Table 2 reports the total FLOPs required for compression and reading (inference), as well as the resulting latency, all measured on a single A100-80GB GPU.

<table>
  <tr>
    <th>Method</th>
    <th>Comp. TFLOPs</th>
    <th>Read. TFLOPs</th>
    <th>Latency (s)</th>
    <th>Speedup</th>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>-</td>
    <td>13.04</td>
    <td>0.95</td>
    <td>1.00x</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>0.54</td>
    <td>1.60</td>
    <td>2.31</td>
    <td>0.41x</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>18.35</td>
    <td>2.05</td>
    <td>12.86</td>
    <td>0.07x</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>15.03</td>
    <td>2.51</td>
    <td>6.78</td>
    <td>0.14x</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>0.27</td>
    <td>1.84</td>
    <td>0.39</td>
    <td>2.43x</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>46.39</td>
    <td>5.06</td>
    <td>0.82</td>
    <td>1.16x</td>
  </tr>
  <tr>
    <td>EXIT (ours)</td>
    <td>35.44</td>
    <td>2.96</td>
    <td>0.88</td>
    <td>1.08x</td>
  </tr>
</table>

Table 2: TFLOPs, latency, and relative speedup of compression methods. Latency is measured on a single A100-80GB GPU. Speedup is relative to the uncompressed baseline (Original Docs).
Table 3: Ablation studies on HQA examining (1) training data composition (Pos, H-Neg, Neg), (2) adaptive vs. fixed-length sentence selection, and (3) the impact of incorporating passage context during classification.

<table>
  <tr>
    <th>Configuration</th>
    <th>EM</th>
    <th>F1</th>
    <th># tokens</th>
  </tr>
  <tr>
    <td>Ours (Pos + H-Neg + Neg)</td>
    <td>31.6</td>
    <td>42.6</td>
    <td>195.1</td>
  </tr>
  <tr>
    <td>Pos + H-Neg</td>
    <td>30.0</td>
    <td>41.3</td>
    <td>286.8</td>
  </tr>
  <tr>
    <td>Pos + Neg</td>
    <td>29.8</td>
    <td>40.9</td>
    <td>404.6</td>
  </tr>
  <tr>
    <td>w/o Adaptive Sentence Selection (2 sends)</td>
    <td>29.4</td>
    <td>40.7</td>
    <td><b>91.0</b></td>
  </tr>
  <tr>
    <td>w/o Adaptive Sentence Selection (4 sends)</td>
    <td>30.2</td>
    <td>41.4</td>
    <td>166.5</td>
  </tr>
  <tr>
    <td>w/o Passage as context</td>
    <td>30.4</td>
    <td>42.3</td>
    <td>157.4</td>
  </tr>
</table>

Although EXIT incurs higher total TFLOPs (38.40) than some competing methods, its design enables parallel sentence-level classification, which significantly reduces end-to-end latency. By contrast, abstractive methods like CompAct and Refiner involve autoregressive decoding, which must be executed sequentially and is subject to memory I/O bottlenecks, leading to much longer runtimes (up to 12.86 seconds).

This reveals a key trade-off: EXIT utilizes more GPU computation, but this computation is efficiently executed in parallel. Since many LLM inference pipelines are memory-bound rather than compute-bound, EXIT takes advantage of under-utilized compute capacity to reduce latency. For example, compared to the uncompressed baseline (0.95s), EXIT achieves a lower latency of 0.88s, despite higher FLOPs.

These results demonstrate that EXIT provides practical efficiency by converting increased computation into latency reduction through parallel execution, offering a better balance between accuracy and response time than autoregressive alternatives.

6.4 Ablation Studies

To better understand how the design choices in EXIT affect its overall performance and efficiency, we conduct ablation studies focusing on three key components: data sampling strategy, adaptive sentence selection, and context-aware extraction. Table 3 summarizes these results.

Data Sampling Strategy. Our training data combines positive, hard negative, and random negative samples to mirror the diversity of real-world retrieval scenarios. Compared to using only subsets of these sample types, the comprehensive strategy improves EM by 1.6 points over using only hard negatives and by 1.8 points over only random negatives. Relying solely on positive and random negatives led to excessive token retention, while depending only on hard negatives diminished the model’s ability to filter out spurious retrieval noise.

Adaptive Sentence Selection. We compare our adaptive selection mechanism to a fixed-length strategy, which limits selection to four sentences. While the fixed-length method reduces token count to 166.5, it lowers EM by 1.4 and F1 by 1.2 points. This highlights the importance of adapting sentence selection based on query complexity and document relevance rather than imposing a static constraint.

Context-Aware Extraction. We evaluate the effect of considering full document context when determining sentence relevance. Removing surrounding context reduces token usage by 38 tokens but decreases EM by 1.2 points, highlighting the importance of contextual awareness in preserving answer accuracy. While context-aware extraction slightly increases token count, it ensures more precise and reliable responses.

These findings confirm that a balanced training data strategy enhances robustness, that adaptive sentence selection ensures efficiency, and that full document consideration preserves accuracy.

6.5 Compressor Model Size and Strategy Impact

Model Size Considerations. Figure 5 presents an ablation study examining how different base models influence EM scores and total latency. Note that all models trained within EXIT achieve superior accuracy compared to uncompressed baselines and fast compression under 2 seconds. Specifically, Gemma-2B, our base classifier model, achieves a favorable balance between effectiveness and efficiency, delivering 31.6 EM points in 0.78s. When moving to the largest model, Gemma-7B, the highest accuracy (32.4 EM) is achieved, yet it also inflates latency to 1.50s, slightly exceeding the time of uncompressed documents. These results suggest that scaling up model parameters can improve

![Ablation on HQA comparing EM scores and latency for different model configurations within EXIT (red dot). CompAct and Original Docs are included as indicators. Experiments on a single A100-80GB GPU.](page_1012_184_410_312.png)

Figure 5: Ablation on HQA comparing EM scores and latency for different model configurations within EXIT (red dot). CompAct and Original Docs are included as indicators. Experiments on a single A100-80GB GPU.
Table 4: Performance comparison across different retrieval and reranking methods on 2WIKI.

<table>
  <tr>
    <th>Method</th>
    <th>EM</th>
    <th>F1</th>
    <th>#tok.</th>
  </tr>
  <tr>
    <th colspan="4">Passage-Level (Top-1)</th>
  </tr>
  <tr>
    <td>bge-m3</td>
    <td>14.6</td>
    <td>22.7</td>
    <td>156.5</td>
  </tr>
  <tr>
    <td>bge-reranker-v2-gemma</td>
    <td>15.0</td>
    <td>22.9</td>
    <td>156.2</td>
  </tr>
  <tr>
    <td>RepLLaMA</td>
    <td>16.0</td>
    <td>24.1</td>
    <td>157.2</td>
  </tr>
  <tr>
    <td>UPR</td>
    <td>9.2</td>
    <td>16.8</td>
    <td>155.3</td>
  </tr>
  <tr>
    <td>Yes/No Ranking Prompt</td>
    <td>13.4</td>
    <td>21.8</td>
    <td>157.5</td>
  </tr>
  <tr>
    <td>Pairwise Ranking Prompt</td>
    <td>13.2</td>
    <td>21.5</td>
    <td>156.2</td>
  </tr>
  <tr>
    <th colspan="4">Sentence-Level (Top-5)</th>
  </tr>
  <tr>
    <td>bge-m3</td>
    <td>16.6</td>
    <td>24.2</td>
    <td>216.9</td>
  </tr>
  <tr>
    <td>bge-reranker-v2-gemma</td>
    <td>18.4</td>
    <td>26.2</td>
    <td>203.2</td>
  </tr>
  <tr>
    <td>RepLLaMA</td>
    <td>14.8</td>
    <td>23.2</td>
    <td>239.0</td>
  </tr>
  <tr>
    <td>UPR</td>
    <td>7.4</td>
    <td>14.6</td>
    <td>148.9</td>
  </tr>
  <tr>
    <td>Yes/No ranking prompt</td>
    <td>14.6</td>
    <td>21.8</td>
    <td>135.6</td>
  </tr>
  <tr>
    <td>Pairwise ranking prompt</td>
    <td>11.6</td>
    <td>19.4</td>
    <td>209.9</td>
  </tr>
  <tr>
    <td>Ours</td>
    <td><b>24.8</b></td>
    <td><b>30.1</b></td>
    <td><b>150.2</b></td>
  </tr>
</table>

performance but may also compromise latency benefits, emphasizing the flexibility of our proposed method by selecting an appropriate model depending on the user requirement.

Abstractive vs. Extractive Compression. Figure 5 also compares our extractive approach, EXIT, against CompAct, a 7B-scale abstractive compressor. Using Mistral-7B as the base model for both methods, EXIT (31.3 EM, 1.46s) significantly outperforms CompAct (30.6 EM, 8.26s) in terms of latency and maintains competitive accuracy. This stark difference underscores that the compression strategy, not the just model size, heavily influences efficiency. By relying on extraction rather than iterative summarization, EXIT capitalizes on a large-scale model to preserve high accuracy without incurring prohibitively long inference time.

6.6 Comparison with Reranking Methods

We compare EXIT with post-retrieval reranking methods in Table 4, including bge-m3 (Chen et al., 2024a), bge-reranker-v2-gemma (Li et al., 2023a), RepLLaMA (Ma et al., 2024), UPR (Sachan et al., 2022), Yes/No ranking prompting (Liang et al., 2023), and Pairwise ranking prompting (Qin et al., 2024). EXIT consistently achieves higher EM and F1 scores, demonstrating its ability to adaptively select query-relevant sentences while filtering out redundancy. By contrast, Top-k re-ranking methods rely on static ranking, often retaining irrelevant content, increasing token count, and diminishing output quality. Moreover, they lack sentence-level context awareness, treating sentences independently without considering their broader document context, which can lead to incoherent or incomplete retrieval. These results underscore EXIT’s advantage in providing a more efficient and contextually relevant input for RAG than other post-retrieval methods.

![Confusion matrix of GPT-4o zero-shot classifier on HotpotQA. The model demonstrates high precision for identifying irrelevant sentences ("No") but shows moderate recall for relevant ones ("Yes").](page_1012_120_388_312.png)

Figure 6: Confusion matrix of GPT-4o zero-shot classifier on HotpotQA. The model demonstrates high precision for identifying irrelevant sentences ("No") but shows moderate recall for relevant ones ("Yes").

6.7 Leveraging LLMs for Pseudo-Annotation

To explore scalable alternatives to manual sentence-level annotations, we evaluated the use of a large language model (GPT-4o) as a zero-shot relevance classifier for generating pseudo-labels. Without any fine-tuning, GPT-4o achieved strong filtering precision, correctly identifying 97% of irrelevant sentences, though with lower recall (54%) for relevant ones (Figure 6). Despite this, EXIT trained with GPT-4o annotations still delivered competitive QA performance (Table 14), highlighting the feasibility of LLM-based pseudo-annotation as a scalable supervision strategy. These findings suggest a promising direction for improving generalization across domains by leveraging LLM-generated training data beyond curated datasets like HQA.

7 Conclusion

We present EXIT, an efficient context compression framework for RAG systems that leverages parallel processing and context-aware, adaptive sentence selection. Our experiments demonstrate that EXIT achieves superior performance across both single-hop and multi-hop QA tasks while maintaining practical inference speeds. Despite being trained only on HQA, EXIT shows a strong zero-shot generalization ability and proves effective across a wide range of open-source models of varying sizes. These results show that parallel extraction with smaller models outperforms larger abstractive methods, making it practical for real-world RAG.
Limitation

This work focuses on RAG pipelines, and its effectiveness in general long-context scenarios like LongBench (Bai et al., 2024) remains future work. Our approach relies on explicit sentence-level annotations for training, which were manually obtained in our experiments. While these could be automated using GPT-4 supervision or reader-derived signals (as discussed in Section C.8), we have not yet explored fully automated annotation strategies—an important direction for future work.

EXIT performs sentence-level extraction, which may introduce limitations when sentences are overly long, noisy, or contain irrelevant details. While our context-aware classifier helps mitigate this, some ambiguity may persist, especially in cases involving complex or entity-heavy sentences. Additionally, although preserving the original sentence order empirically yields the best performance, we observed that removing intermediate sentences can occasionally lead to unnatural or incoherent reconstructions. Addressing this may require future improvements such as sentence decomposition, coherence-aware selection, or lightweight sentence refinement mechanisms.

We also focus on a single-step RAG setting, excluding iterative or recursive retrieval (Shao et al., 2023; Trivedi et al., 2023; Khattab et al., 2022). However, EXIT is orthogonal to these approaches and can be integrated by compressing retrieved content at each step.

Finally, while EXIT avoids the latency bottlenecks of autoregressive generation, further efficiency gains could be achieved through architectural optimizations such as prefix caching or Radix-Attention (Zheng et al., 2024), which remain open areas for future exploration.

Ethics Statement

This work enhances RAG-based QA without generating new content beyond what is retrieved. However, biases and inaccuracies in the source documents can still propagate through our compression process. Ensuring the reliability, fairness, and proper curation of underlying corpora is essential for ethical deployment. Future efforts should integrate bias detection, provenance tracking, and user-centric evaluations to promote more transparent and equitable real-world applications.

Acknowledgments

This work was supported by the Institute for Information and communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) (No. 2022-0-00010, Development of Korean sign language translation service technology for the deaf in medical environment).

References

Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. Longbench: A bilingual, multi-task benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 3119–3137. Association for Computational Linguistics.

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024a. BGE m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. CoRR, abs/2402.03216.

Tong Chen, Hongwei Wang, Sihao Chen, Wenhao Yu, Kaixin Ma, Xinran Zhao, Hongming Zhang, and Dong Yu. 2024b. Dense X retrieval: What retrieval granularity should we use? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pages 15159–15177. Association for Computational Linguistics.

Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics.

Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, and Dongyan Zhao. 2024. xrag: Extreme context compression for retrieval-augmented generation with one token. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024.

Alexis Chevalier, Alexander Wetting, Anirudh Ajith, and Danqi Chen. 2023. Adapting language models to compress contexts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 3829–3846. Association for Computational Linguistics.

Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, and Michael
Collins. 2021. Decontextualization: Making sentences stand-alone. Trans. Assoc. Comput. Linguistics, 9:447–461.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aoob Yang, Archi Mitra, Archie Sravanakumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolidakis, Damien Alionsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esibou, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Neil, Grégoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vraneš, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasudevanwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. 2024. The llama 3 herd of models. CoRR, abs/2407.21783.

Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furui Wei. 2024. In-context autoencoder for context compression in a large language model. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.

Michael Günther, Isabelle Mohr, Bo Wang, and Han Xiao. 2024. Late chunking: Contextual chunk embeddings using long-context embedding models. CoRR, abs/2409.04701.

Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 6609–6625. International Committee on Computational Linguistics.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.

Taeho Hwang, Soyeong Jeong, Sukmin Cho, SeungYoon Han, and Jong Park. 2024. DSLR: Document refinement with sentence-level re-ranking and reconstruction to enhance retrieval-augmented generation. In Proceedings of the 3rd Workshop on Knowledge Augmented Methods for NLP, pages 73–92, Bangkok, Thailand. Association for Computational Linguistics.

Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised dense information retrieval with contrastive learning. Trans. Mach. Learn. Res., 2022.

Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 7036–7050. Association for Computational Linguistics.

Soyeong Jeong, Jinheon Baek, Sung Ju Hwang, and Jong Park. 2023. Phrase retrieval for open domain conversational question answering with conversational dependency modeling via contrastive learning. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 6019–6031. Association for Computational Linguistics.

Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. Lmllingua: Compressing prompts for accelerated inference of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 13358–13376. Association for Computational Linguistics.

Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2024. Longllmllingua: Accelerating and enhancing llms in long context scenarios via prompt compression. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 1658–1677. Association for Computational Linguistics.

Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 1601–1611. Association for Computational Linguistics.

Vladimir Karpukhin, Barlas Ogun, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for
open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 6769–6781. Association for Computational Linguistics.

Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.

Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. 2022. Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive NLP. CoRR, abs/2212.14024.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenneth Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics, 7:452–466.

Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, and Danqi Chen. 2021. Learning dense representations of phrases at scale. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021 (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 6634–6647. Association for Computational Linguistics.

Yuho Lee, Taewon Yun, Jason Cai, Hang Su, and Hwanjun Song. 2024. Unisumeval: Towards unified, fine-grained, multi-dimensional summarization evaluation for llms. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 3941–3960. Association for Computational Linguistics.

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7871–7880. Association for Computational Linguistics.

Patrick S. H. Lewis, Ethan Perez, Aleksandra Pikutus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.

Chaofan Li, Zheng Liu, Shitao Xiao, and Yingxia Shao. 2023a. Making large language models A better foundation for dense retrieval. CoRR, abs/2312.15503.

Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix X. Yu, and Sanjiv Kumar. 2023b. Large language models with controllable working memory. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 1774–1793. Association for Computational Linguistics.

Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023c. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6449–6464. Association for Computational Linguistics.

Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023d. Compressing context to enhance inference efficiency of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6342–6353. Association for Computational Linguistics.

Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023e. Compressing context to enhance inference efficiency of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6342–6353. Association for Computational Linguistics.

Zhonghao Li, Xuming Hu, Aiwei Liu, Kening Zheng, Sirui Huang, and Hui Xiong. 2024. Refiner: Restructure retrieval content efficiently to advance question-answering capabilities. CoRR, abs/2406.11357.

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel J. Orr, Lucia Zheng, Mert Yüksekgönül, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. 2023. Holistic evaluation of language models. Trans. Mach. Learn. Res., 2023.

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Trans. Assoc. Comput. Linguistics, 12:157–173.
Kun Luo, Zheng Liu, Shitao Xiao, Tong Zhou, Yubo Chen, Jun Zhao, and Kang Liu. 2024. Landmark embedding: A chunking-free embedding method for retrieval augmented long-context large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 3268–3281. Association for Computational Linguistics.

Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2024. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2024, Washington DC, USA, July 14-18, 2024, pages 2421–2425. ACM.

Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Taffi, Léonard Husseinot, Akanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Cristian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, and et al. 2024. Gemma: Open models based on gemini research and technology. CoRR, abs/2403.08295.

Timo Möller, Anthony Reina, Raghavan Jayakumar, and Malte Pietsch. 2020. COVID-QA: A question answering dataset for COVID-19. In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. Online. Association for Computational Linguistics.

Jesse Mu, Xiang Li, and Noah D. Goodman. 2023. Learning to compress prompts with gist tokens. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.

Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Ranking sentences for extractive summarization with reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 1747–1759. Association for Computational Linguistics.

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, December 9, 2016, volume 1773 of CEUR Workshop Proceedings. CEUR-WS.org.

Rodrigo Frassetto Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. arXiv, 1901.04085, abs/1901.04085.

OpenAI. 2023. GPT-4 technical report. arXiv:2303.08774, abs/2303.08774.

Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. Imllingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 963–981. Association for Computational Linguistics.

Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2024. Yam: Effective context window extension of large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.

Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. 2023. Large language models are effective text rankers with pairwise ranking prompting. arXiv:2306.17563, abs/2306.17563.

Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. 2024. Large language models are effective text rankers with pairwise ranking prompting. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 1504–1518. Association for Computational Linguistics.

Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. Trans. Assoc. Comput. Linguistics, 11:1316–1331.

Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 3505–3506. ACM.

Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen M. Voorhees, Lucy Lu Wang, and William R. Hersh.
2021. Searching for scientific evidence in a pandemic: An overview of TREC-COVID. J. Biomed. Informatics, 121:103865.

Stephen E. Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333–389.

Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Agbabyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022. Improving passage retrieval with zero-shot question generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 3781–3797. Association for Computational Linguistics.

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 1073–1083. Association for Computational Linguistics.

Min Joon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P. Parikh, Ali Farhadi, and Hannaneh Hajishirzi. 2019. Real-time open-domain question answering with dense-sparse phrase index. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 4430–4441. Association for Computational Linguistics.

Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 9248–9274. Association for Computational Linguistics.

Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 31210–31227. PMLR.

Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2024. REPLUG: retrieval-augmented black-box language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 8371–8384. Association for Computational Linguistics.

Hwanjun Song, Hang Su, Igor Shalyminov, Jason Cai, and Saab Mansour. 2024a. Finesure: Fine-grained summarization evaluation using llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 906–922. Association for Computational Linguistics.

Hwanjun Song, Taewon Yun, Yuho Lee, Gihun Lee, Jason Cai, and Hang Su. 2024b. Learning to summarize from llm-generated feedback. CoRR, abs/2410.13116.

Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-hop questions via single-hop question composition. Trans. Assoc. Comput. Linguistics, 10:539–554.

Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 10014–10037. Association for Computational Linguistics.

George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R. Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopoulos, Yannis Almirantis, John Pavlopoulos, Nicolas Baskiotis, Patrick Gallinari, Thierry Artières, Axel-Cyrille Ngonga Ngomo, Norman Heino, Éric Gaussier, Liliana Barrio-Alvers, Michael Schroeder, Ion Androutsopoulos, and Georgios Palioras. 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinform., 16:138:1–138:28.

Zhiruo Wang, Jun Araki, Zhengbao Jiang, Md. Rizwan Parvez, and Graham Neubig. 2023. Learning to filter context for retrieval-augmented generation. arXiv:2311.08377, abs/2311.08377.

David Wingate, Mohammad Shoeybi, and Taylor Sorensen. 2022. Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models. In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 5621–5634. Association for Computational Linguistics.

Zhenyu Wu, Chao Shen, and Meng Jiang. 2024. Instructing large language models to identify and ignore irrelevant conditions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pages 6799–6819. Association for Computational Linguistics.
Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Su. 2024a. Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, pages 7655–7671. Association for Computational Linguistics.

Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. 2024b. RULE: reliable multimodal RAG for factuality in medical vision language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pages 1081–1093. Association for Computational Linguistics.

Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2024. RE-COMP: Improving retrieval-augmented LMs with context compression and selective augmentation. In The Twelfth International Conference on Learning Representations.

Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Pengcheng He, Chenguang Zhu, and Michael Zeng. 2023. LMQOS: A large-scale dataset for query-focused summarization. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 14764–14776. Association for Computational Linguistics.

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 2369–2380. Association for Computational Linguistics.

Chanwoong Yoon, Taewho Lee, Hyeon Hwang, Min-byul Jeong, and Jaewoo Kang. 2024. Compact: Compressing retrieved documents actively for question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, pages 21424–21439. Association for Computational Linguistics.

Weijia Zhang, Jia-Hong Huang, Svitlana Vakulenko, Yumo Xu, Thilina Rajapakse, and Evangelos Kanoulas. 2024. Beyond relevant documents: A knowledge-intensive approach for query-focused summarization using large language models. In Pattern Recognition - 27th International Conference, ICPR 2024, Kolkata, India, December 1-5, 2024, Proceedings, Part XIX, volume 15319 of Lecture Notes in Computer Science, pages 89–104. Springer.

Zheng Zhao, Shay B. Cohen, and Bonnie Webber. 2020. Reducing quantity hallucinations in abstractive summarization. CoRR, abs/2009.13312.

Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng. 2024. Sglang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024.

Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022. Towards a unified multi-dimensional evaluator for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2023–2038. Association for Computational Linguistics.
Appendix

In the Appendix, we provide additional implementation details and present supplementary results and analyses not covered in the main text.

A  More Related Work

A.1 Information Retrieval

Advancements in information retrieval (IR) have improved retrieval efficiency, granularity, and ranking quality. Encoder-based models, such as DPR (Karpukhin et al., 2020) and Contriever (Izard et al., 2022), have been widely adopted for retrieval tasks. More recently, M3-Embedding (Chen et al., 2024a) unifies dense, multi-vector, and sparse retrieval via self-knowledge distillation, enabling retrieval across different text granularities.

Decoder-only LLMs have emerged as strong rerankers, leveraging large-scale data for improved ranking at the cost of computational complexity. LLaRA (Li et al., 2023a) and RePLlama (Ma et al., 2024) employ dense retrieval and multi-stage ranking, while UPR (Sachan et al., 2022) and ranking prompting methods (Liang et al., 2023; Qin et al., 2024) refine retrieval through generation-based approaches. However, these methods focus on reordering passages rather than compressing retrieved content.

Research on retrieval granularity (Seo et al., 2019; Lee et al., 2021; Jeong et al., 2023; Chen et al., 2024b; Hwang et al., 2024) explores balancing precision and contextual coherence. While fine-grained retrieval enhances specificity, excessive fragmentation can distort meaning (Choi et al., 2021). Retrieval and ranking aim to maximize recall but do not address post-retrieval redundancy. EXIT operates beyond retrieval and ranking, focusing on adaptive post-retrieval context compression, which selectively filters content to optimize both efficiency and relevance in RAG.

A.2 Summarization

Summarization methods are broadly categorized as extractive and abstractive. Extractive summarization retains factual consistency by selecting key sentences but often suffers from redundancy and coherence issues (Cheng and Lapata, 2016; Narayan et al., 2018). Abstractive summarization improves fluency and readability but is computationally intensive and prone to hallucination (See et al., 2017; Lewis et al., 2020a). Despite these challenges, its ability to generate concise outputs has driven interest in its application (Song et al., 2024b; Lee et al., 2024; Song et al., 2024a).

Query-focused summarization refines document content based on relevance to a given query, removing unrelated information while preserving key details (Zhang et al., 2024; Xu et al., 2023). However, traditional summarization methods process single documents and prioritize completeness and faithfulness (Song et al., 2024a,b). By contrast, post-retrieval compression in RAG requires query-adaptive filtering across multiple retrieved documents. EXIT fills this gap by providing a dynamic and query-adaptive compression strategy, distinguishing between relevant, marginal, and irrelevant content while maintaining both efficiency over latency and answer accuracy.

B  More Implementation Details

This section describes our training environment, data composition, and prompt templates. All experiments were conducted on an NVIDIA A100-SXM4-80GB GPU cluster. Training was performed on a single GPU with gradient accumulation.

B.1 Training Configuration

We employed LoRA (Hu et al., 2022) to fine-tune the compressor model, enabling parameter-efficient training while preserving performance. The model was trained with the following hyperparameters:
• Batch size: 8 per device
• Gradient accumulation steps: 8
• Learning rate: 1e-5
• Weight decay: 0.1
• Warmup ratio: 0.03
• Training epochs: 1
• Optimizer: paged_adamw_8bit
• Quantization: 4-bit with float16 precision
• LoRA configuration: Rank = 64, Scaling = 32, Dropout = 0.05

B.2 Model Selection and Training Time

Model selection was guided by validation loss. Training was conducted on our cluster, requiring approximately 90 hours.

B.3 Data Processing

We used SpaCy to segment documents into sentences. Table 5 shows the composition of the training and validation sets derived from HQA. The training set contains 427K sentences, including 213K positive (Pos), 107K hard-negative (H-Neg),
Table 5: Statistics of the training dataset constructed from HQA. Positive (Pos) sentences are required for the correct answer, Hard-Neg (H-Neg) sentences appear in the same passages but lack crucial evidence, and Neg sentences come from unrelated queries. Counts are in thousands (K).

<table>
  <tr>
    <th>Split</th>
    <th>Pos</th>
    <th>H-Neg</th>
    <th>Neg</th>
    <th>Total</th>
  </tr>
  <tr>
    <td>Train</td>
    <td>213K</td>
    <td>107K</td>
    <td>107K</td>
    <td>427K</td>
  </tr>
  <tr>
    <td>Valid</td>
    <td>2.4K</td>
    <td>1.2K</td>
    <td>1.2K</td>
    <td>4.8K</td>
  </tr>
</table>

and 107K negative (Neg) instances. The validation set includes 4.8K sentences with a similar distribution. This balanced composition ensures the classifier encounters diverse retrieval scenarios during training.

B.4 Inference Settings

For inference, we set:
• Temperature: 0.0
• Top-p: 1.0
• vLLM version: v0.5.5
• Relevance threshold (\( \tau \)): 0.5

B.5 Prompt Templates

Table 6: A prompt template for document compression.

<table>
  <tr>
    <th colspan="2">Compression Prompt Template</th>
  </tr>
  <tr>
    <td>Query:</td>
    <td>{query}</td>
  </tr>
  <tr>
    <td>Full context:</td>
    <td>{original passage}</td>
  </tr>
  <tr>
    <td>Sentence:</td>
    <td>{sentence}</td>
  </tr>
  <tr>
    <td colspan="2">Is this sentence useful in answering the query? Answer only “Yes” or “No”.</td>
  </tr>
</table>

Full prompt templates for compression and QA tasks are provided in Tables 6 and 7, respectively.

C Additional Experimental Results and Analyses

Table 21 shows detailed results for each dataset and model configuration under zero-shot QA prompts, using both Top-5 and Top-20 retrieval. For the few-shot QA prompts, Table 22 summarizes the results, where we randomly selected five training examples per dataset as demonstrations. Table 23 provides comprehensive token count statistics, and Table 24 breaks down end-to-end latency.

C.1 Performance with Sparse Retrieval

To assess EXIT’s robustness with different retrieval architectures, we evaluate it with BM25, a sparse retrieval method. Table 8 compares EXIT’s performance on HQA (in-domain) and 2WIKI (out-of-domain) under Top-5 and Top-20 retrieval settings.

With Top-5 retrieval, EXIT shows notable gains on HQA, improving EM (33.4 vs. 28.2) and F1 (44.5 vs. 39.1) over the uncompressed baseline. Although RECOMP-Abst performs best on 2WIKI (25.0 EM, 30.6 F1), EXIT remains competitive (24.4 EM, 29.1 F1).

EXIT’s advantages grow with Top-20 retrieval. On HQA, EXIT outperforms all baselines, improving EM by 4.0 points (35.2 vs. 31.2) and F1 by

Table 7: A prompt template used by the reader model for the QA task.

QA Prompt Template

Context information is below.

{context}

Given the context information and not prior knowledge, answer the query. Do not provide any explanation.
Query: {query}
Answer:

Table 8: Performance comparison between in-domain (HQA) and out-of-domain (2WIKI) datasets using BM25 as the retriever model. Best results are highlighted in bold, and second best results are underlined.

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="2">HQA</th>
    <th colspan="2">2WIKI</th>
  </tr>
  <tr>
    <th>EM ↑</th>
    <th>F1 ↑</th>
    <th>EM ↑</th>
    <th>F1 ↑</th>
  </tr>
  <tr>
    <td colspan="6"><i>Top-5 Documents</i></td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>Abs.</td>
    <td>28.2</td>
    <td>39.1</td>
    <td>755.8 (100.0)</td>
    <td>19.6</td>
    <td>25.9</td>
    <td>789.7 (100.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>Abs.</td>
    <td>27.8</td>
    <td>39.2</td>
    <td><b>63.0 (83.3)</b></td>
    <td><b>25.0</b></td>
    <td><b>30.6</b></td>
    <td><b>55.7 (7.1)</b></td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>Abs.</td>
    <td>30.6</td>
    <td>41.2</td>
    <td>76.9 (10.2)</td>
    <td>19.6</td>
    <td>29.1</td>
    <td>71.0 (9.0)</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>Abs.</td>
    <td>30.8</td>
    <td>42.3</td>
    <td>84.0 (11.1)</td>
    <td>19.6</td>
    <td>27.8</td>
    <td>62.9 (8.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>Ext.</td>
    <td>28.2</td>
    <td>37.5</td>
    <td>93.1 (12.3)</td>
    <td>11.8</td>
    <td>19.1</td>
    <td>99.1 (12.5)</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>Ext.</td>
    <td>28.6</td>
    <td>39.7</td>
    <td>230.3 (30.5)</td>
    <td>22.2</td>
    <td>27.4</td>
    <td>236.6 (30.0)</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td>Ext.</td>
    <td><b>33.4</b></td>
    <td><b>44.5</b></td>
    <td>177.8 (23.5)</td>
    <td><b>24.4</b></td>
    <td><b>29.1</b></td>
    <td>138.2 (17.5)</td>
  </tr>
  <tr>
    <td colspan="8"><i>Top-20 Documents</i></td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>Abs.</td>
    <td>31.2</td>
    <td>42.5</td>
    <td>3009.5 (100.0)</td>
    <td>23.0</td>
    <td>30.6</td>
    <td>3132.5 (100.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>Abs.</td>
    <td>29.0</td>
    <td>39.7</td>
    <td><b>69.7 (23.3)</b></td>
    <td>22.8</td>
    <td>28.2</td>
    <td><b>52.3 (1.7)</b></td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>Abs.</td>
    <td>31.6</td>
    <td>43.0</td>
    <td>109.5 (3.6)</td>
    <td>20.4</td>
    <td>28.7</td>
    <td>113.2 (3.6)</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>Abs.</td>
    <td>28.4</td>
    <td>38.8</td>
    <td>136.1 (4.5)</td>
    <td>18.8</td>
    <td>26.9</td>
    <td>108.4 (3.5)</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>Ext.</td>
    <td>28.4</td>
    <td>37.0</td>
    <td>93.5 (3.1)</td>
    <td>11.2</td>
    <td>19.3</td>
    <td>96.7 (3.1)</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>Ext.</td>
    <td>31.0</td>
    <td>40.7</td>
    <td>558.0 (18.5)</td>
    <td>23.6</td>
    <td>28.3</td>
    <td>593.7 (19.0)</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td>Ext.</td>
    <td><b>35.2</b></td>
    <td><b>46.9</b></td>
    <td>411.3 (13.7)</td>
    <td><b>27.2</b></td>
    <td><b>32.4</b></td>
    <td>312.7 (10.0)</td>
  </tr>
</table>
Table 9: Performance comparison between in-domain (HQA) and out-of-domain (2WIKI) datasets using GPT-4o as the reader model. Best results are highlighted in bold, and second best results are underlined.

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="3">HQA</th>
    <th colspan="3">2WIKI</th>
  </tr>
  <tr>
    <th>EM</th>
    <th>F1</th>
    <th>Tokens (%)</th>
    <th>EM</th>
    <th>F1</th>
    <th>Tokens (%)</th>
  </tr>
  <tr>
    <td colspan="8"><b>Top-5 Documents</b></td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>-</td>
    <td>37.2</td>
    <td>48.6</td>
    <td>735.3 (100.0)</td>
    <td>31.2</td>
    <td>25.3</td>
    <td>764.5 (100.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>Abs.</td>
    <td>29.4</td>
    <td>40.2</td>
    <td><b>62.8 (8.5)</b></td>
    <td>23.8</td>
    <td>27.8</td>
    <td><b>53.5 (7.0)</b></td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>Abs.</td>
    <td>37.4</td>
    <td>48.0</td>
    <td>74.3 (10.1)</td>
    <td>30.0</td>
    <td>33.6</td>
    <td>67.6 (8.8)</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>Abs.</td>
    <td>35.8</td>
    <td>47.5</td>
    <td>71.4 (9.7)</td>
    <td>27.8</td>
    <td>32.6</td>
    <td>54.2 (7.1)</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>Ext.</td>
    <td>32.4</td>
    <td>41.7</td>
    <td>87.1 (11.8)</td>
    <td>25.6</td>
    <td>28.6</td>
    <td>93.6 (12.2)</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>Ext.</td>
    <td>34.2</td>
    <td>45.2</td>
    <td>22.3 (13.0)</td>
    <td>30.6</td>
    <td>34.5</td>
    <td>230.5 (30.2)</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td>Ext.</td>
    <td><b>38.2</b></td>
    <td><b>50.4</b></td>
    <td>19.1 (2.6)</td>
    <td><b>31.8</b></td>
    <td><b>35.8</b></td>
    <td>145.6 (19.0)</td>
  </tr>
  <tr>
    <td colspan="8"><b>Top-20 Documents</b></td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>-</td>
    <td><b>39.6</b></td>
    <td><b>51.8</b></td>
    <td>2940.5 (100.0)</td>
    <td><b>40.0</b></td>
    <td><b>43.8</b></td>
    <td>3066.2 (100.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>Abs.</td>
    <td>33.6</td>
    <td>44.2</td>
    <td>62.7 (2.1)</td>
    <td>26.6</td>
    <td>32.1</td>
    <td><b>48.9 (1.6)</b></td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>Abs.</td>
    <td>33.0</td>
    <td>43.7</td>
    <td>106.0 (3.6)</td>
    <td>23.0</td>
    <td>27.3</td>
    <td>105.1 (3.4)</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>Abs.</td>
    <td>31.8</td>
    <td>41.5</td>
    <td>130.6 (4.4)</td>
    <td>31.0</td>
    <td>35.5</td>
    <td>100.3 (3.3)</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>Ext.</td>
    <td>31.2</td>
    <td>39.6</td>
    <td>86.2 (2.9)</td>
    <td>23.2</td>
    <td>27.2</td>
    <td>91.0 (3.0)</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>Ext.</td>
    <td>38.8</td>
    <td>49.4</td>
    <td>549.7 (18.7)</td>
    <td>35.4</td>
    <td>39.6</td>
    <td>581.5 (19.0)</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td>Ext.</td>
    <td><b>38.2</b></td>
    <td><b>50.4</b></td>
    <td>43.0 (13.4)</td>
    <td><b>31.8</b></td>
    <td><b>35.8</b></td>
    <td>348.9 (11.8)</td>
  </tr>
</table>

4.4 points (46.9 vs. 42.5) compared to using uncompressed documents. On 2WIKI, EXIT achieves the highest scores (27.2 EM, 32.4 F1), confirming its generalizability across domains and retrieval strategies.

C.2 Performance with Proprietary Model

We further examine EXIT’s effectiveness using GPT-4o as the reader. Table 9 compares performance on HQA (in-domain) and 2WIKI (out-of-domain), along with compression rates.

For Top-5 retrieval, EXIT attains the best accuracy on HQA (38.2 EM, 50.4 F1) while retaining only 26.0% of tokens. This surpasses the uncompressed baseline (37.2 EM, 48.6 F1) with a 74% token reduction. On 2WIKI, EXIT maintains leading accuracy (31.8 EM, 35.8 F1) while using just 19.0% of the original tokens.

Under Top-20 retrieval, where uncompressed documents benefit from greater coverage, EXIT still achieves competitive accuracy with substantially fewer tokens. On HQA, EXIT closely matches the uncompressed EM score (39.4 vs. 39.6) while using only 15.4% of tokens. Although RECOMP variants compress more aggressively, they suffer marked performance drops. LongLLMLingua performs similarly to EXIT but retains more tokens (18.7% vs. 15.4%).

These findings illustrate EXIT’s ability to balance performance and efficiency, making it valuable for API-based proprietary models where token costs and accuracy both matter.

Table 10: Evaluation of EXIT and baselines in specialized domains. BioASQ evaluates biomedical QA using PubMed documents, and COVID-QA focuses on COVID-specific QA over the TREC-COVID corpus.

<table>
  <tr>
    <th colspan="4">BioASQ (Biomedical QA)</th>
  </tr>
  <tr>
    <th>Method</th>
    <th>EM</th>
    <th>F1</th>
    <th>Avg. #Tokens</th>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>57.2</td>
    <td>68.6</td>
    <td>1791.49</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>54.6</td>
    <td>63.8</td>
    <td>37.72</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>58.0</td>
    <td>68.2</td>
    <td>94.81</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>58.8</td>
    <td>68.5</td>
    <td>229.22</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>55.8</td>
    <td>65.0</td>
    <td>109.72</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>58.2</td>
    <td>66.4</td>
    <td>364.06</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>58.6</b></td>
    <td><b>67.1</b></td>
    <td><b>132.28</b></td>
  </tr>
  <tr>
    <th colspan="4">COVID-QA (COVID-19 QA)</th>
  </tr>
  <tr>
    <th>Method</th>
    <th>EM</th>
    <th>F1</th>
    <th>Avg. #Tokens</th>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>3.8</td>
    <td>16.0</td>
    <td>1775.66</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>2.2</td>
    <td>11.5</td>
    <td>17.94</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>3.4</td>
    <td>14.6</td>
    <td>95.65</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>3.8</td>
    <td>14.6</td>
    <td>201.38</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>1.4</td>
    <td>11.4</td>
    <td>103.11</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>2.2</td>
    <td>13.5</td>
    <td>353.03</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>5.8</b></td>
    <td><b>17.2</b></td>
    <td><b>129.33</b></td>
  </tr>
</table>

C.3 Evaluation in Specialized Domains

To assess the generalizability of EXIT beyond general-domain datasets, we conducted additional evaluations on two specialized QA benchmarks: 1) BioASQ (Tsatsaronis et al., 2015), a biomedical QA dataset using the PubMed corpus (14.9M documents), 2) COVID-QA (Möller et al., 2020), a COVID-specific QA benchmark based on the TREC-COVID collection (Roberts et al., 2021) (171K documents).

Although our classifier was trained solely on HQA, a general-domain dataset, EXIT performs competitively or better than existing baselines in both specialized domains. Table 10 shows that EXIT achieves high EM and F1 scores while substantially reducing token counts. On COVID-QA, EXIT notably outperforms all baselines in EM and F1 despite the domain shift.

These results suggest that EXIT’s context-aware extraction strategy generalizes well, preserving key evidence without requiring domain-specific fine-tuning. This highlights EXIT’s robustness across diverse retrieval scenarios.

C.4 Evaluation on Complex Multi-hop Reasoning

To further assess EXIT’s ability to handle complex reasoning tasks, we evaluated it on the MuSiQue dataset (Trivedi et al., 2022), which contains multi-hop questions requiring deeper reasoning (3–4
Table 11: Evaluation of EXIT and baselines on the MuSiQue dataset on Llama3-8B reader.

<table>
  <tr>
    <th>Method</th>
    <th>EM</th>
    <th>F1</th>
    <th>Latency (s)</th>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>3.8</td>
    <td>10.3</td>
    <td>1.1</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>4.5</td>
    <td>11.8</td>
    <td>2.5</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>4.1</td>
    <td>11.0</td>
    <td>8.5</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>4.5</td>
    <td>11.3</td>
    <td>7.6</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>4.3</td>
    <td>10.1</td>
    <td><b>0.6</b></td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>4.0</td>
    <td>10.7</td>
    <td>0.9</td>
  </tr>
  <tr>
    <td><b>EXIT (Ours)</b></td>
    <td><b>4.6</b></td>
    <td><b>11.3</b></td>
    <td><b>0.9</b></td>
  </tr>
</table>

hops). While our main experiments focused on standard benchmarks like HQA and 2WIKI, MuSiQue provides a valuable test of generalization under more challenging conditions.

As shown in Table 11, EXIT achieves competitive performance on MuSiQue. Notably, it maintains low latency while matching or outperforming other extractive and abstractive baselines in EM and F1 scores. These results demonstrate EXIT’s robustness in handling longer reasoning chains without incurring the high latency costs typically associated with abstractive methods.

These findings reinforce EXIT’s ability to scale to more complex, multi-hop QA settings while maintaining an efficient trade-off between latency and performance.

C.5 Latency Measurement and Analysis

To ensure the reliability and reproducibility of our latency results, we conducted five independent runs for each compression method and report both the mean and standard deviation of end-to-end latency (measured in seconds per sample). End-to-end latency includes both compression time and reading/inference time, measured from the moment a request is issued to the moment the answer is returned by the model.

For consistency, all experiments using the 8B reader were run on a single A100-80GB SXM GPU, while those using the 70B reader were executed on four A100-80GB SXM GPUs. To minimize hardware-induced noise, we ensured that no unrelated processes were active during measurement and maintained fixed hyperparameters and model configurations across all runs.

As shown in Table 12, our method EXIT achieves a strong balance between latency and accuracy. While abstractive methods such as CompAct and Refiner incur significant latency (12.86s and 6.78s respectively, with the 8B reader), EXIT completes inference in under 1 second (0.88s) on average. Even with the 70B reader, EXIT remains efficient with 3.86s total latency, significantly faster than the original uncompressed baseline (8.08s), while also improving QA performance.

These results reinforce EXIT’s practicality for real-world deployment, particularly in latency-sensitive applications.

C.6 Impact of Threshold \( \tau \)

We analyze EXIT’s sensitivity to the relevance threshold \( \tau \). Figure 7 shows EXIT’s performance across various \( \tau \) values.

EXIT remains stable over a wide threshold range, with strong results between \( \tau = 0.3–0.5 \). At \( \tau = 0.3 \), EXIT reaches 25.2 EM using only 25% of the tokens (195.82 vs. 780.95 for the baseline), a substantial improvement over the original documents (18.0 EM). F1 scores also remain consistently higher than the baseline (30.21–30.73 vs. 25.74).

Even under extreme compression (\( \tau = 0.9, 7.9\% \) of tokens), EXIT achieves better accuracy (24.0 EM, 29.44 F1) than the uncompressed documents. Conversely, a lenient threshold (\( \tau = 0.1 \)) retains more tokens but still provides benefits, demonstrating that EXIT effectively identifies crucial content under varying conditions.

This robustness across thresholds gives practitioners flexibility to adjust the compression-accuracy trade-off without severely impacting performance.

C.7 Effect of Classifier Training

To evaluate the impact of fine-tuning the classifier in EXIT, we compared our trained model with a frozen zero-shot version using Gemma-2B (without any task-specific fine-tuning). We tested both variants on an in-domain dataset (HQA) and an out-of-domain dataset (2WIKI).

As shown in Table 13, the zero-shot classifier already improves over the uncompressed baseline, indicating the strength of relevance-based filtering even without training. However, our fine-tuned classifier consistently outperforms the zero-shot variant in both EM and F1 scores while achieving significantly greater token reduction—demonstrating that supervised training yields both better accuracy and more efficient compression.

C.8 Adaptability of EXIT to Different Supervision Signals

A key strength of EXIT is its ability to integrate different classifiers for sentence selection without
Table 12: Average end-to-end latency (seconds per sample) across different compression methods. Results are the mean and standard deviation (Std) over five runs. Experiments for the 8B reader were conducted on a single A100-80GB SXM GPU, and for the 70B reader on four A100-80GB SXM GPUs.

<table>
  <tr>
    <th rowspan="2">8B Reader</th>
    <th colspan="2">Compression</th>
    <th colspan="2">Reading</th>
    <th colspan="2">Total</th>
  </tr>
  <tr>
    <th>Method</th>
    <th>Avg</th>
    <th>Std</th>
    <th>Avg</th>
    <th>Std</th>
    <th>Avg</th>
    <th>Std</th>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>–</td>
    <td>–</td>
    <td>0.9481</td>
    <td>0.0056</td>
    <td>0.9481</td>
    <td>0.0056</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>1.9967</td>
    <td>0.0159</td>
    <td>0.3177</td>
    <td>0.0008</td>
    <td>2.3144</td>
    <td>0.0167</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>12.4682</td>
    <td>0.0030</td>
    <td>0.3930</td>
    <td>0.0006</td>
    <td>12.8612</td>
    <td>0.0036</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>6.3881</td>
    <td>0.0332</td>
    <td>0.3877</td>
    <td>0.0012</td>
    <td>6.7758</td>
    <td>0.0344</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>0.0279</td>
    <td>0.0022</td>
    <td>0.3668</td>
    <td>0.0040</td>
    <td>0.3947</td>
    <td>0.0062</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>0.3721</td>
    <td>0.0050</td>
    <td>0.4526</td>
    <td>0.0004</td>
    <td>0.8247</td>
    <td>0.0054</td>
  </tr>
  <tr>
    <td><b>EXIT (Ours)</b></td>
    <td><b>0.4263</b></td>
    <td><b>0.0031</b></td>
    <td><b>0.4529</b></td>
    <td><b>0.0028</b></td>
    <td><b>0.8792</b></td>
    <td><b>0.0059</b></td>
  </tr>
  <tr>
    <th colspan="2">70B Reader</th>
    <th colspan="2">Compression</th>
    <th colspan="2">Reading</th>
    <th colspan="2">Total</th>
  </tr>
  <tr>
    <td>Method</td>
    <td>Avg</td>
    <td>Std</td>
    <td>Avg</td>
    <td>Std</td>
    <td>Avg</td>
    <td>Std</td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>–</td>
    <td>–</td>
    <td>8.0835</td>
    <td>0.0072</td>
    <td>8.0835</td>
    <td>0.0072</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>1.8621</td>
    <td>0.0171</td>
    <td>2.3210</td>
    <td>0.0113</td>
    <td>4.1831</td>
    <td>0.0284</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>15.1999</td>
    <td>0.0132</td>
    <td>2.6932</td>
    <td>0.0007</td>
    <td>17.8931</td>
    <td>0.0139</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>8.3520</td>
    <td>0.0248</td>
    <td>3.0993</td>
    <td>0.0017</td>
    <td>11.4513</td>
    <td>0.0265</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>0.0329</td>
    <td>0.0069</td>
    <td>2.8288</td>
    <td>0.0031</td>
    <td>2.8617</td>
    <td>0.0100</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>0.4788</td>
    <td>0.0127</td>
    <td>3.7121</td>
    <td>0.0012</td>
    <td>4.1909</td>
    <td>0.0139</td>
  </tr>
  <tr>
    <td><b>EXIT (Ours)</b></td>
    <td><b>0.3253</b></td>
    <td><b>0.0086</b></td>
    <td><b>3.5341</b></td>
    <td><b>0.0022</b></td>
    <td><b>3.8594</b></td>
    <td><b>0.0108</b></td>
  </tr>
</table>

![Bar charts showing changes in EM, F1 score, and token count as the threshold τ for retaining sentences is adjusted.](page_101_1012_1242_370.png)

Figure 7: Changes in EM, F1 score, and token count as the threshold \( \tau \) for retaining sentences is adjusted.

being constrained to manually annotated datasets. To explore this flexibility, we evaluate an alternative approach where relevance scores are derived from GPT-4o without explicit fine-tuning. As shown in Table 14, EXIT maintains strong performance, outperforming or closely matching baselines such as LongLLMLingua and CompAct. This demonstrates that EXIT can leverage diverse supervision signals while remaining adaptable to different scoring mechanisms. Moreover, these results suggest the potential of utilizing large-scale pseudo-labeled data to further refine EXIT’s training, enhancing scalability without relying strictly on human-labeled datasets. This adaptability highlights EXIT’s robustness and practical applicability across various retrieval settings.

C.9 Analysis of Classification Performance Across Negative Sample Types

Table 15 presents EXIT’s sentence-level classification performance, broken down by negative sample type. EXIT achieves 0.92 F1 for both positive (“Yes”) and negative (“No”) classes overall, indicating a balanced ability to identify essential and non-essential sentences.

The model excels at filtering random negatives (0.95 F1), effectively discarding irrelevant content. With hard negatives (topically related but not essential), EXIT still performs well (0.89 F1 for positive, 0.88 for negative), handling nuanced relevance distinctions.

These results highlight EXIT’s adaptability and confirm its suitability for real-world RAG scenarios
Table 13: Comparison of EXIT using a zero-shot (frozen Gemma-2B) vs. fine-tuned classifier on HotpotQA (HQA) and 2WikiMultiHopQA (2WIKI).

<table>
  <tr>
    <th>Dataset</th>
    <th>Method</th>
    <th>EM</th>
    <th>F1</th>
  </tr>
  <tr>
    <td rowspan="3">2WIKI</td>
    <td>Original Docs</td>
    <td>18.0</td>
    <td>25.7</td>
  </tr>
  <tr>
    <td>EXIT (Zero-shot)</td>
    <td>22.4</td>
    <td>27.9</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>24.8</b></td>
    <td><b>30.1</b></td>
  </tr>
  <tr>
    <td rowspan="3">HQA</td>
    <td>Original Docs</td>
    <td>29.2</td>
    <td>40.2</td>
  </tr>
  <tr>
    <td>EXIT (Zero-shot)</td>
    <td>31.4</td>
    <td><b>43.1</b></td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>31.6</b></td>
    <td>42.6</td>
  </tr>
</table>

<table>
  <tr>
    <th>Dataset</th>
    <th>Method</th>
    <th>Avg. #Tokens</th>
  </tr>
  <tr>
    <td rowspan="3">2WIKI</td>
    <td>Original Docs</td>
    <td>764.47</td>
  </tr>
  <tr>
    <td>EXIT (Zero-shot)</td>
    <td>731.35</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>145.62</b></td>
  </tr>
  <tr>
    <td rowspan="3">HQA</td>
    <td>Original Docs</td>
    <td>735.24</td>
  </tr>
  <tr>
    <td>EXIT (Zero-shot)</td>
    <td>727.87</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>191.11</b></td>
  </tr>
</table>

Table 14: Comparison of EXIT with baselines on QA tasks on HQA. EXIT (GPT-4o) refers to a variant using GPT-4o as a zero-shot relevance classifier, while EXIT (Ours) represents our proposed method trained with supervised data.

<table>
  <tr>
    <th>Method</th>
    <th>EM</th>
    <th>F1</th>
    <th>#token</th>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>29.2</td>
    <td>40.2</td>
    <td>735.2</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>25.2</td>
    <td>34.8</td>
    <td>89.7</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>19.8</td>
    <td>24.7</td>
    <td>55.5</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>27.4</td>
    <td>40.2</td>
    <td>227.1</td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>29.4</td>
    <td>40.9</td>
    <td>76.3</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>28.8</td>
    <td>40.7</td>
    <td>73.4</td>
  </tr>
  <tr>
    <td>EXIT (GPT-4o)</td>
    <td>30.4</td>
    <td>41.9</td>
    <td>84.2</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td>
    <td><b>31.6</b></td>
    <td><b>42.6</b></td>
    <td>191.0</td>
  </tr>
</table>

where both overtly irrelevant and subtly extraneous content must be managed.

C.10 Classification Performance

To better understand the effectiveness of our context-aware relevance classifier, we report row-normalized confusion matrices for both in-domain (HQA) and out-of-domain (2WIKI) datasets, as shown in Figure 8. On HQA, the classifier displays a perfectly balanced ability to recognize both relevant (“Yes”) and irrelevant (“No”) sentences, achieving over 90% precision and recall in each category. While, on the 2WIKI dataset, the classifier exhibits a slight drop in recall for “Yes” sentences, it still shows a strong classification ability with over 70% recall and 90% precision.

These results confirm that the classifier performs robustly in its training domain and generalizes reasonably well to unseen queries, though we leave narrowing this discrepancy as a valuable future research direction.

Table 15: Classification performance for Yes/No labels.

<table>
  <tr>
    <th colspan="4">Overall</th>
  </tr>
  <tr>
    <th>Class</th>
    <th>Precision ↑</th>
    <th>Recall ↑</th>
    <th>F1-Score ↑</th>
  </tr>
  <tr>
    <td>Yes</td>
    <td>0.91</td>
    <td><b>0.93</b></td>
    <td>0.92</td>
  </tr>
  <tr>
    <td>No</td>
    <td><b>0.93</b></td>
    <td>0.91</td>
    <td>0.92</td>
  </tr>
  <tr>
    <th colspan="4">Hard Negative</th>
  </tr>
  <tr>
    <td>Yes</td>
    <td>0.86</td>
    <td><b>0.93</b></td>
    <td>0.89</td>
  </tr>
  <tr>
    <td>No</td>
    <td><b>0.93</b></td>
    <td>0.84</td>
    <td>0.88</td>
  </tr>
  <tr>
    <th colspan="4">Negative</th>
  </tr>
  <tr>
    <td>Yes</td>
    <td><b>0.96</b></td>
    <td>0.93</td>
    <td>0.95</td>
  </tr>
  <tr>
    <td>No</td>
    <td>0.93</td>
    <td><b>0.96</b></td>
    <td>0.95</td>
  </tr>
</table>

![Confusion matrices for context-aware relevance classification on HQA (in-domain) and 2WIKI (out-of-domain)](page_1012_1042_377_246.png)

Figure 8: Confusion matrices (row-normalized) for context-aware relevance classification on HQA (in-domain) and 2WIKI (out-of-domain).

C.11 Classification Performance under Ablation Settings

C.11.1 Analysis of Training Data Composition

Figure 9 presents row-normalized confusion matrices comparing classification performance across three training data configurations: Ours (Pos+H-Neg+Neg), Pos+H-Neg, and Pos+Neg. Under the Ours setup, the classifier displays a balanced ability to identify both “Yes” (relevant) and “No” (irrelevant) sentences, achieving an F1-score of 0.92 for both classes. By contrast, excluding one type of negative sample (Pos+H-Neg or Pos+Neg) reduces overall robustness, evidenced by declines in both accuracy and class-wise F1 scores. For instance, the Pos+Neg configuration struggles to maintain balance, accurately identifying “Yes” instances but misclassifying a substantial number of “No” cases. These results confirm that incorporating a comprehensive mix of positive, hard-negative, and random-negative samples leads to more reliable and contextually aware sentence selection, thereby improving the classifier’s performance in practical retrieval-augmented QA scenarios.
Figure 9: Row-normalized confusion matrices for classification performance under different training data conditions: Ours (Pos+H-Neg+Neg), Pos+H-Neg, and Pos+Neg. Each matrix compares the predicted (“Yes”/“No”) labels against the actual labels.

Figure 10: Row-normalized confusion matrices comparing classification performance with (left) and without (right) contextual information on HQA. The availability of context improves the model’s ability to accurately distinguish relevant (“Yes”) from non-relevant (“No”) sentences.

C.11.2 Impact of Context on Classification Performance

We evaluate the classifier’s performance with and without broader passage-level context. Figure 10 shows that including context maintains over 90% precision and recall for both “Yes” and “No” classes. Without context, precision and recall decline, weakening the distinction between relevant and irrelevant sentences. This emphasizes the importance of incorporating passage-level context for accurately identifying answer-critical information.

C.12 Sentence Reordering Strategy Analysis

One potential concern with extractive compression is that removing many sentences may lead to unnatural or ambiguous contexts when the remaining content is reassembled. In EXIT, we preserve the original sentence order during reconstruction to maintain document coherence, following prior work (Hwang et al., 2024). However, we acknowledge that this decision could lead to suboptimal flow when large gaps exist between selected sentences.

To assess this design choice, we conducted an ablation study comparing four sentence reordering strategies: 1) Random: randomly shuffles selected sentences, 2) Ascending: sorts by relevance score (low to high), 3) Descending: sorts by relevance score (high to low), 4) Original order (EXIT): preserves sentence sequence from the source document

Table 16 shows that maintaining the original order consistently yields the best EM and F1 scores on HQA. This confirms that preserving the source structure helps retain contextual flow, even when intermediate content is omitted. We include this result to support our design and to acknowledge the known limitations of extractive methods in handling discourse coherence.

<table>
  <tr>
    <th>Reordering Strategy</th>
    <th>EM</th>
    <th>F1</th>
  </tr>
  <tr>
    <td>Random</td>
    <td>30.20</td>
    <td>41.30</td>
  </tr>
  <tr>
    <td>Ascending (score-based)</td>
    <td>30.43</td>
    <td>41.44</td>
  </tr>
  <tr>
    <td>Descending (score-based)</td>
    <td>30.33</td>
    <td>41.16</td>
  </tr>
  <tr>
    <td><b>Original order (EXIT)</b></td>
    <td><b>30.61</b></td>
    <td><b>41.49</b></td>
  </tr>
</table>

Table 16: Ablation results comparing sentence reordering strategies in EXIT on HotpotQA. “Random” shuffles selected sentences, while “Ascending” and “Descending” sort by relevance scores. “Original order” maintains the sentence sequence from the original document.

<table>
  <tr>
    <th>Training Data</th>
    <th>EM ↑</th>
    <th>F1 ↑</th>
    <th># token ↓</th>
  </tr>
  <tr>
    <td>HQA</td>
    <td>31.6</td>
    <td>42.6</td>
    <td>195.1</td>
  </tr>
  <tr>
    <td>2WIKI</td>
    <td>29.2</td>
    <td>40.3</td>
    <td>135.3</td>
  </tr>
  <tr>
    <td>2WIKI+HQA</td>
    <td>30.6</td>
    <td>42.0</td>
    <td>232.2</td>
  </tr>
</table>

Table 17: Training data ablation comparison. Best results per metric are highlighted in bold.

C.13 Training Data Ablation Analysis

Table 17 compares models trained on HQA, 2WIKI, or both. Training solely on HQA yields the highest EM and F1 (31.6 EM, 42.6 F1) with moderate token usage. By contrast, 2WIKI training improves compression but lowers accuracy (29.2 EM, 40.3 F1). Combining datasets does not surpass HQA alone.

This finding suggests that data quality and structure matter more than quantity. HQA’s annotations appear particularly effective for learning robust compression strategies that generalize well, validating our choice to use it as the primary training dataset.
C.14 Case Studies
To illustrate how EXIT’s extractive compression strategy improves both accuracy and readability, we present qualitative examples and comparisons with other compression methods.

Original Documents vs. Ours. In Table 18, the original documents contain the correct answer (“Custard”) but also include distracting information (“Eggnog”). Despite having the necessary evidence, the reader fails to produce the correct answer, likely due to this distractor. By contrast, EXIT (Ours) filters out irrelevant details, drastically reduces input length, and retains only the essential context needed to answer the query accurately. As a result, the reader confidently generates the correct answer (“Custard”).

In a second scenario (Table 19), the original documents retrieve multiple documents related to “Wagner” or “sci-fi” series but fail to provide any content explicitly linking James Belushi to the correct 90s sci-fi series, resulting in an incorrect prediction. Surprisingly, EXIT removes all retrieved context entirely, providing the reader with no additional information. Under this no-context condition, the reader relies solely on its internal knowledge and, in this case, correctly identifies “Wild Palms.” While this outcome indicates a form of hallucination or model bias—since the answer emerges without external supporting evidence—it also demonstrates EXIT’s capability to avoid misleading context. By eliminating irrelevant or confusing documents, EXIT can sometimes allow the model’s internal knowledge to surface, leading to correct answers even in the absence of any retrieved information.

Comparisons with Other Methods. Table 20 compares EXIT with several competing compression approaches. CompAct preserves some relevant information but introduces hallucinations, causing the reader to claim that it cannot find the correct answer. Refiner omits the crucial entity required to answer the query, demonstrating how abstractive compressors may inadvertently remove key content. By contrast, EXIT avoids hallucinations and retains the answer’s entity in a concise, coherent form.

LongLLMLingua’s token-level filtering approach yields unreadable text and removes the essential “Romania” entity, preventing the reader from generating the correct answer. EXIT, on the other hand, maintains semantic coherence and includes the correct entity, allowing the reader to produce the correct answer without interference.

These case studies highlight the advantages of EXIT: it eliminates distractors, preserves critical entities, and maintains semantic integrity. Consequently, the reader consistently arrives at correct answers with fewer tokens and no hallucinations.
Table 18: Case study comparing compressed contexts and answers between Original Docs and Ours.

<table>
  <tr>
    <th>Query</th>
    <th>Original Docs</th>
    <th>Ours</th>
  </tr>
  <tr>
    <td>Crème Anglaise is the French version of which English dessert item?</td>
    <td>[1] Creme anglaise<br>Crème anglaise Crème anglaise (French for '"English cream"') is a light pouring Custard used as a dessert cream or sauce. It is a mix of sugar, egg yolks, oil, and hot milk ...<br>[2] Creme anglaise<br>However, the ice cream base is much thicker and has various flavourings. The American South it is known as '"Custard." It can be served like Eggnog during the Christmas season. Other names include the French terms '"crème à l'anglaise"' ("English-style cream") and '"crème française"' ("French cream"). Crème anglaise Crème anglaise (French for '"English cream"') is a light pouring Custard used as a dessert cream or sauce. It is a mix of sugar, egg yolks, oil, and hot milk often flavoured with vanilla. Its name may derive from the prevalence of sweet Custards in English desserts. The cream is made by<br>[3] Creme anglaise<br>...<br>[4] Custard lemon. '"Crème pâtissière"' is a key ingredient in many French desserts including mille-feuille (or Napoleons) and filled tarts. It is also used in Italian pastry and sometimes in Boston cream pie. The thickening of the Custard is caused by the combination of egg and cornstarch. Corn flour or flour thicken at 100 °C and as such many recipes instruct the pastry cream to be boiled. In a traditional Custard such as a '"crème anglaise"', where egg is used alone as a thickener, boiling results in the over cooking and subsequent 'curling' of the Custard; however, in a pastry cream, starch<br>[5] Cremeschnitte<br>usually pure thick Custard, less commonly combined with meringue (whipped egg whites and sugar) creme. A similar recipe with only meringue filling is called Šampita. In Australia, the dish is more commonly known as a 'vanilla slice'. Cremeschnitte A cremeschnitte (,,,,,, ) is a chantilly and Custard cream cake dessert popular in several Central European countries. There are many regional variations, but they all include a puff pastry base and Custard cream. In Slovenia, kremna rezina is commonly associated with the town of Bled, an Alpine tourist destination in northwestern Slovenia. The recipe</td>
    <td>[1] Creme anglaise<br>Crème anglaise Crème anglaise (French for '"English cream"') is a light pouring Custard used as a dessert cream or sauce.<br>[2] Creme anglaise<br>Other names include the French terms '"crème à l'anglaise"' ("English-style cream") and '"crème française"' ("French cream"). Crème anglaise Crème anglaise (French for '"English cream"') is a light pouring Custard used as a dessert cream or sauce.<br>[3] Creme anglaise<br>Alternatively, it can be drunk as a dessert on its own, for example in '"Île flottante"' ("floating island"); the cream is poured into a bowl with a piece of meringue ("blancs en neige") floated on top along with praline. It can also be used as a base for desserts such as ice cream or crème brûlée.<br>[4] Custard<br>"Crème pâtissière"' is a key ingredient in many French desserts including mille-feuille (or Napoleons) and filled tarts.</td>
  </tr>
  <tr>
    <th>Answer</th>
    <td>Custard</td>
    <td>Custard</td>
  </tr>
  <tr>
    <th>Predict</th>
    <td>Eggnog</td>
    <td>Custard</td>
  </tr>
</table>
Table 19: Case study comparing compressed contexts and answers between Original Docs and Ours. Despite containing no relevant context, Ours method predicts the correct answer, indicating a hallucination scenario.

<table>
  <tr>
    <th>Original Docs</th>
    <th>Ours</th>
  </tr>
  <tr>
    <th>Query</th>
    <td>Which 90s sci fi series with James Belushi was based on Bruce Wagner’s comic strip of the same name?</td>
    <td><i>No context</i> (completely pruned)</td>
  </tr>
  <tr>
    <th>Context</th>
    <td>[1] Michael I. Wagner<br>patterned his character after Wagner’s mannerisms and physical behavior. The series ran on Thursday nights in the Spring of 1988 during the same time slot as NBC’s “The Cosby Show”, and with that competition could not attract a sufficient audience to get renewed for the following season.<br>...<br>Wagner helped develop and write the Bochco animated series "Capitol Critters", he also wrote and served as supervising producer<br>[2] Michael I. Wagner<br>Steven Bochco and several of his projects. Wagner was asked by ABC in 1987 to help develop a new science fiction series, "Probe", a light-hearted series about a scientific crime fighter named Austin James.<br>...<br>Parker Stevenson, who played the lead character, stated in a later interview that he<br>[3] John Wagner<br>the mid-1990s Wagner worked on a number of licensed properties for Dark Horse Comics in the US, including "Aliens", "Star Wars" – notably solo stories starring Boba Fett and the comics strand of the multimedia project '" – and '".<br>...<br>It was nominated for the Angoulême International Comics Festival Prize for Scenario in 2006. In 2000 Wagner<br>[4] Martin Wagner (artist)<br>Martin Wagner (artist) Martin Wagner (born April 27, 1966) is an American artist, cartoonist, and filmmaker.<br>...<br>. His production schedule became increasingly protracted and he ceased publishing the series altogether following issue No. 12 in 1994. In 1996 he made a<br>[5] Wired (film)<br>"L.A. Law", "Murphy Brown", and "Seinfeld"), Chiklis gained fame for portraying the lead roles of Commissioner Tony Scali on the ABC police drama "The Commish" (1991-1996), and LAPD Detective Vic Mackey on the FX police drama "The Shield" (2002-2008) and as well as Marvel superhero Ben "The Thing" Grimm in the films "Fantastic Four" (2005) and '" (2007).<br>...<br>Wired (film) Wired is a 1989 biographical film of comedian and actor John Belushi, directed by Larry Peerce, and adapted from the 1984 book of the same name by "Washington"</td>
    <td></td>
  </tr>
  <tr>
    <th>Answer</th>
    <td>Wild Palms</td>
    <td>Wild Palms</td>
  </tr>
  <tr>
    <th>Predict</th>
    <td>Probe</td>
    <td>Wild Palms</td>
  </tr>
</table>
Table 20: Combined case studies comparing compressed contexts and answers across different methods.

<table>
  <tr>
    <th>Case Study 1</th>
    <th>CompAct</th>
    <th>Ours</th>
  </tr>
  <tr>
    <td><b>Query</b></td>
    <td>Where was there a series of explosions and fire on 23 October 1989 (killing 23 employees and injuring 314), an explosion on 23 June 1999 (killing two men and injuring three), and an explosion and fire on 27 March 2000 (causing 1 death and 71 injuries)?</td>
    <td>[1] Phillips disaster of 1989<br>On October 23, 1989, at 1400 Jefferson Road, Pasadena, Texas, a powerful explosion killed 23 and injured 314. A powerful and devastating explosion and fire ripped through the HCC, killing 23 persons—all working at the facility—and injuring 314 other (185 Phillips 66 employees and 129 contract employees).</td>
  </tr>
  <tr>
    <td><b>Context</b></td>
    <td>The Flixborough disaster was an explosion at a chemical plant near Flixborough, England on 1 June 1974. The incident started on October 23, 1989, at 1400 Jefferson Road, Pasadena, Texas.</td>
    <td></td>
  </tr>
  <tr>
    <td><b>Answer</b></td>
    <td>Pasadena, Texas</td>
    <td></td>
  </tr>
  <tr>
    <td><b>Predict</b></td>
    <td>I couldn’t find any information about the specific incidents you mentioned. The Flixborough disaster occurred on 1 June 1974.</td>
    <td>Pasadena, Texas</td>
  </tr>
  <tr>
    <th>Case Study 2</th>
    <th>Refiner</th>
    <th>Ours</th>
  </tr>
  <tr>
    <td><b>Query</b></td>
    <td>Who said that there are only three true sports, bullfighting, mountaineering, and motor racing, the rest are merely games?</td>
    <td>[1] Extreme sport<br>usually, but wrongly, attributed to Ernest Hemingway. The phrase is: There are only three sports: bullfighting, motor racing, and mountaineering; all the rest are merely games.</td>
  </tr>
  <tr>
    <td><b>Context</b></td>
    <td>1.1. Extreme sport<br>There are only three sports: bullfighting, motor racing, and mountaineering; all the rest are merely games.</td>
    <td></td>
  </tr>
  <tr>
    <td><b>Answer</b></td>
    <td>Ernest Hemingway</td>
    <td></td>
  </tr>
  <tr>
    <td><b>Predict</b></td>
    <td>There is no information provided about who said that.</td>
    <td>Ernest Hemingway</td>
  </tr>
  <tr>
    <th>Case Study 3</th>
    <th>LongLLMLingua</th>
    <th>Ours</th>
  </tr>
  <tr>
    <td><b>Query</b></td>
    <td>Which was the only Eastern Bloc country to participate in the 1984 LA Olympics?</td>
    <td>[1] 1984 Summer Olympics boycott<br>The boycott involved 14 Eastern Bloc countries and allies, led by the Soviet Union, which initiated the boycott on May 8, 1984.<br>...<br>[3] Summer Olympic Games<br>Eastern Bloc that did attend the 1984 Olympics. Although a boycott led by the Soviet Union depleted the field, 140 NOCs took part. Without Eastern European countries, the 1984 Games were dominated by the host. The Games were also the first time mainland China participated.<br>...<br>[5] 1984 Summer Olympics boycott<br>However, no threat to Eastern Bloc athletes was discovered, and the athletes from the Eastern Bloc country that did attend the 1984 games—Romania—encountered no problems.</td>
  </tr>
  <tr>
    <td><b>Context</b></td>
    <td>984 Summer Olympics the majority of Soviet Bloc will not participate Games.ausescu’s Roman expected.984 Summer boyce The boyce of the 1984 Summer in Los Angeles years after U.S.-led boycott of the 1980 Summer Olympics In The boycott involved 14 Bloc countries and allies, led by the Soviet Union, which init boycott on May 8, 1984. Booting countries organized major event, the Friendship Games, in July and August 1984 Although the boycott by the Soviet Union affected number Olympic Sum were perhaps the first games of a new era to make a profit. Although a boycott led by the Soviet Union depleted the field in certain sports, 140 National Olympic Committees took part, which was a record at the time. Again, without the participation of the Eastern European countries, the 1984 Games were dominated by their host country. The Games were also the first time mainland China (People’s Republic) participated. According to British journalist Andrew Jennings, a KGB colonel stated that the agency’s officers had posed as anti-doping authorities from</td>
    <td></td>
  </tr>
  <tr>
    <td><b>Answer</b></td>
    <td>Romania</td>
    <td></td>
  </tr>
  <tr>
    <td><b>Predict</b></td>
    <td>China</td>
    <td>Romania</td>
  </tr>
</table>
Table 21: Zero-shot QA prompt evaluation of compressor performance across different Top-k scenarios, models, and datasets, measured by EM, F1, and inference latency (Lat.). 8B reader experiments were conducted on a single A100-80GB GPU, while 70B reader experiments utilized 4 A100-80GB GPUs in parallel. Best results for each dataset are highlighted in bold. Second best results are highlighted in underline.

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="3">NQ</th>
    <th colspan="3">TQA</th>
    <th colspan="3">HQA</th>
    <th colspan="3">2WIKI</th>
    <th colspan="3">AVG.</th>
  </tr>
  <tr>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
    <th>EM ↑</th><th>F1 ↑</th><th>Lat. ↓</th>
  </tr>
  <tr>
    <th colspan="15">Llama-3.1-8B-Instruct</th>
  </tr>
  <tr>
    <th colspan="15">Top-5 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>34.6</td><td>47.1</td><td>1.0</td><td>58.8</td><td>68.6</td><td>0.9</td><td>28.1</td><td>38.6</td><td>1.0</td><td>16.1</td><td>24.9</td><td>1.1</td><td>34.4</td><td>44.8</td><td>1.0</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>31.3</td><td>43.2</td><td>1.6</td><td>55.9</td><td>65.7</td><td>1.4</td><td>26.5</td><td>37.0</td><td>2.2</td><td>22.7</td><td>29.1</td><td>2.1</td><td>34.1</td><td>43.7</td><td>1.8</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>32.9</td><td>44.6</td><td>8.5</td><td>58.1</td><td>67.7</td><td>8.8</td><td>28.8</td><td>39.8</td><td>8.3</td><td>16.8</td><td>26.0</td><td>8.1</td><td>34.2</td><td>44.5</td><td>8.4</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>32.9</td><td>45.0</td><td>28.1</td><td>59.2</td><td>68.9</td><td>10.9</td><td>28.8</td><td>40.0</td><td>6.9</td><td>16.8</td><td>25.4</td><td>6.4</td><td>34.4</td><td>44.8</td><td>13.1</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>34.6</td><td>44.6</td><td>0.5</td><td>56.5</td><td>65.1</td><td>0.4</td><td>23.4</td><td>32.8</td><td>0.4</td><td>11.2</td><td>19.6</td><td>0.6</td><td>31.4</td><td>40.5</td><td>0.5</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>30.2</td><td>41.5</td><td>0.9</td><td>59.4</td><td>68.0</td><td>0.8</td><td>28.0</td><td>38</td><td>0.8</td><td>21.5</td><td>27.4</td><td>0.9</td><td>34.8</td><td>43.7</td><td>0.9</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td><b>35.9</b></td><td><b>47.8</b></td><td><u>0.8</u></td><td><b>60.8</b></td><td><b>69.9</b></td><td><u>0.7</u></td><td><b>30.6</b></td><td><b>41.5</b></td><td><u>0.8</u></td><td><b>24.2</b></td><td><b>30.8</b></td><td><u>0.9</u></td><td><b>37.9</b></td><td><b>47.5</b></td><td><u>0.8</u></td>
  </tr>
  <tr>
    <th colspan="15">Top-20 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>36.6</td><td>49.5</td><td>3.4</td><td>62.0</td><td>71.7</td><td>2.9</td><td>29.9</td><td>40.5</td><td>2.9</td><td>18.8</td><td>27.9</td><td>3.2</td><td>36.8</td><td>47.4</td><td>3.1</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>26.9</td><td>38.3</td><td>1.7</td><td>57.3</td><td>66.6</td><td>1.9</td><td>26.8</td><td>37.1</td><td>2.4</td><td>22.7</td><td>28.8</td><td>2.6</td><td>33.4</td><td>42.7</td><td>2.2</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>33.8</td><td>45.4</td><td>26.1</td><td>57.8</td><td>67.5</td><td>24.5</td><td>28.9</td><td>39.6</td><td>27.5</td><td>16.7</td><td>24.6</td><td>32.2</td><td>34.3</td><td>44.3</td><td>27.6</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>30.1</td><td>41.4</td><td>28.7</td><td>57.6</td><td>67.0</td><td>44.2</td><td>26.6</td><td>37.3</td><td>29.6</td><td>16.3</td><td>24.9</td><td>10.8</td><td>32.7</td><td>42.6</td><td>28.3</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>32.8</td><td>42.6</td><td>0.6</td><td>55.5</td><td>63.6</td><td>0.4</td><td>22.2</td><td>31.2</td><td>0.5</td><td>10.0</td><td>18.3</td><td>0.7</td><td>30.1</td><td>38.9</td><td>0.6</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>33.4</td><td>45.1</td><td>2.8</td><td>62.4</td><td>71.2</td><td>2.7</td><td>31.2</td><td>41.4</td><td>2.8</td><td>24.1</td><td>30.1</td><td>2.9</td><td>37.8</td><td>46.9</td><td>2.8</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td><b>38.1</b></td><td><b>50.8</b></td><td><u>1.8</u></td><td><b>62.8</b></td><td><b>72.0</b></td><td><u>1.7</u></td><td><b>32.9</b></td><td><b>44.0</b></td><td><u>1.8</u></td><td><b>25.5</b></td><td><b>32.3</b></td><td><u>2.0</u></td><td><b>39.8</b></td><td><b>49.8</b></td><td><u>1.8</u></td>
  </tr>
  <tr>
    <th colspan="15">Llama-3.1-70B-Instruct</th>
  </tr>
  <tr>
    <th colspan="15">Top-5 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>35.6</td><td>48.0</td><td>8.6</td><td>65.1</td><td>73.9</td><td>7.7</td><td>33.7</td><td>44.5</td><td>8.3</td><td>20.8</td><td>28.3</td><td>9.1</td><td>38.8</td><td>48.7</td><td>8.4</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>34.1</td><td>47.0</td><td>4.5</td><td>61.3</td><td>70.6</td><td>3.3</td><td>30.3</td><td>40.8</td><td>4.4</td><td>24.2</td><td>30.3</td><td>4.2</td><td>37.5</td><td>47.2</td><td>4.1</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>34.1</td><td>45.4</td><td>11.9</td><td>62.6</td><td>71.1</td><td>11.7</td><td>33.8</td><td>44.1</td><td>11.0</td><td>20.5</td><td>27.4</td><td>11.6</td><td>37.8</td><td>47.0</td><td>11.5</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>35.3</td><td>47.1</td><td>42.5</td><td>64.3</td><td>73.0</td><td>18.3</td><td>33.8</td><td>44.7</td><td>14.6</td><td>21.2</td><td>28.0</td><td>11.2</td><td>38.7</td><td>48.2</td><td>21.6</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td><b>35.8</b></td><td><b>45.3</b></td><td><u>2.5</u></td><td>63.5</td><td>71.0</td><td><u>2.2</u></td><td>27.6</td><td>36.7</td><td><u>2.9</u></td><td>13.8</td><td>19.3</td><td><u>3.3</u></td><td>35.2</td><td>43.1</td><td><u>2.7</u></td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>32.2</td><td>44.0</td><td>4.4</td><td>66.7</td><td>75.2</td><td>3.9</td><td>34.1</td><td>45.3</td><td>4.0</td><td>28.3</td><td><b>34.8</b></td><td>4.3</td><td>40.3</td><td>49.8</td><td>4.1</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td><b>36.9</b></td><td><b>49.4</b></td><td><u>3.9</u></td><td><b>67.3</b></td><td><b>75.9</b></td><td><u>3.1</u></td><td><b>37.0</b></td><td><b>48.3</b></td><td><u>3.3</u></td><td><b>28.6</b></td><td><b>34.5</b></td><td><u>3.5</u></td><td><b>42.5</b></td><td><b>52.0</b></td><td><u>3.5</u></td>
  </tr>
  <tr>
    <th colspan="15">Top-20 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>39.5</td><td>52.5</td><td>25.8</td><td><b>69.1</b></td><td><b>77.6</b></td><td>24.9</td><td>38.5</td><td>50.0</td><td>25.3</td><td>28.8</td><td><b>36.8</b></td><td>28.1</td><td><b>44.0</b></td><td><b>54.2</b></td><td>26</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>31.5</td><td>45.1</td><td>4.5</td><td>63.4</td><td>72.2</td><td>3.7</td><td>31.3</td><td>41.8</td><td>4.8</td><td>25.4</td><td>30.7</td><td><u>4.8</u></td><td>37.9</td><td>47.4</td><td><u>4.5</u></td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>33.9</td><td>45.1</td><td>30.8</td><td>61.7</td><td>70.0</td><td>28.1</td><td>31.7</td><td>40.9</td><td>32.0</td><td>18.5</td><td>23.5</td><td>36.5</td><td>36.4</td><td>44.9</td><td>31.9</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>32.6</td><td>43.5</td><td>37.9</td><td>62.9</td><td>71.4</td><td>48.9</td><td>31.9</td><td>42.2</td><td>33.0</td><td>22.7</td><td>28.7</td><td>12.8</td><td>37.5</td><td>46.5</td><td>33.2</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>33.6</td><td>42.7</td><td><u>2.4</u></td><td>63.1</td><td>70.3</td><td><u>2.2</u></td><td>25.6</td><td>34.5</td><td><u>2.9</u></td><td>12.2</td><td>17.4</td><td><u>3.3</u></td><td>33.6</td><td>41.2</td><td><u>2.7</u></td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>34.5</td><td>46.4</td><td>10.9</td><td>68.2</td><td>76.9</td><td>10.4</td><td>36.7</td><td>48.4</td><td>10.6</td><td>29.8</td><td>36.5</td><td>11.0</td><td>42.3</td><td>52.1</td><td>10.7</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td><b>39.4</b></td><td><b>52.6</b></td><td><u>5.1</u></td><td><b>68.7</b></td><td><b>77.3</b></td><td><u>4.3</u></td><td><b>38.6</b></td><td><b>50.2</b></td><td><u>4.7</u></td><td><b>30.0</b></td><td><b>36.3</b></td><td><u>4.8</u></td><td><b>44.2</b></td><td><b>54.1</b></td><td><u>4.7</u></td>
  </tr>
</table>
Table 22: Few-shot QA prompt evaluation measured by EM and F1. Best results for each dataset are highlighted in bold. Second best results are highlighted in underline.

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="2">NQ</th>
    <th colspan="2">TQA</th>
    <th colspan="2">HQA</th>
    <th colspan="2">2WIKI</th>
    <th colspan="2">AVG.</th>
  </tr>
  <tr>
    <th>EM↑</th><th>F1↑</th>
    <th>EM↑</th><th>F1↑</th>
    <th>EM↑</th><th>F1↑</th>
    <th>EM↑</th><th>F1↑</th>
    <th>EM↑</th><th>F1↑</th>
  </tr>
  <tr>
    <th colspan="12">Llama-3.1-8B-Instruct</th>
  </tr>
  <tr>
    <th colspan="12">Top-5 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>36.9</td><td>48.8</td><td>61.5</td><td>70.3</td><td>29.6</td><td>39.9</td><td>22.2</td><td>29.1</td><td>37.5</td><td>47.0</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>33.9</td><td>45.1</td><td>57.8</td><td>66.7</td><td>27.0</td><td>37.2</td><td>26.2</td><td>32.2</td><td>36.2</td><td>45.3</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>35.0</td><td>46.3</td><td>60.3</td><td>69.2</td><td>30.2</td><td>40.6</td><td>23.9</td><td>31.0</td><td>37.3</td><td>46.8</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>34.4</td><td>46.0</td><td>61.0</td><td>70.0</td><td>29.6</td><td>39.9</td><td>23.4</td><td>30.0</td><td>37.1</td><td>46.5</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>35.9</td><td>45.8</td><td>58.5</td><td>66.1</td><td>25.9</td><td>35.4</td><td>21.2</td><td>27.5</td><td>35.4</td><td>43.7</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>30.7</td><td>41.8</td><td>60.8</td><td>68.8</td><td>27.3</td><td>37.1</td><td>23.0</td><td>28.8</td><td>35.5</td><td>44.1</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td><td>Ext.</td><td>35.8</td><td>47.4</td><td>61.0</td><td>69.8</td><td>29.7</td><td>40.3</td><td>25.9</td><td>32.0</td><td>38.1</td><td>47.4</td>
  </tr>
  <tr>
    <th colspan="12">Top-20 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>39.2</td><td>51.4</td><td>64.2</td><td>73.2</td><td>30.6</td><td>40.8</td><td>24.8</td><td>32.4</td><td>39.7</td><td>49.4</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>30.2</td><td>40.7</td><td>59.3</td><td>67.6</td><td>27.4</td><td>37.7</td><td>26.8</td><td>32.4</td><td>35.9</td><td>44.6</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>35.6</td><td>47.0</td><td>60.1</td><td>69.2</td><td>30.7</td><td>40.6</td><td>21.0</td><td>28.3</td><td>36.9</td><td>46.3</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>32.2</td><td>43.1</td><td>59.6</td><td>68.6</td><td>27.8</td><td>37.9</td><td>22.8</td><td>29.4</td><td>35.6</td><td>44.7</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>34.2</td><td>43.7</td><td>57.5</td><td>64.9</td><td>24.6</td><td>33.8</td><td>19.8</td><td>26.0</td><td>34.0</td><td>42.1</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>33.8</td><td>45.2</td><td>63.8</td><td>72.1</td><td>31.0</td><td>41.1</td><td>25.3</td><td>31.6</td><td>38.5</td><td>47.5</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td><td>Ext.</td><td>38.8</td><td>50.8</td><td>63.4</td><td>72.3</td><td>32.2</td><td>43.1</td><td>26.6</td><td>32.9</td><td>40.3</td><td>49.8</td>
  </tr>
  <tr>
    <th colspan="12">Llama-3.1-70B-Instruct</th>
  </tr>
  <tr>
    <th colspan="12">Top-5 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>39.5</td><td>51.8</td><td>68.3</td><td>76.5</td><td>36.0</td><td>46.7</td><td>31.4</td><td>37.5</td><td>43.8</td><td>53.1</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>38.1</td><td>50.3</td><td>63.4</td><td>72.4</td><td>30.8</td><td>41.2</td><td>27.8</td><td>33.4</td><td>40.0</td><td>49.3</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>37.9</td><td>49.7</td><td>67.7</td><td>76.0</td><td>36.8</td><td>47.5</td><td>32.2</td><td>38.7</td><td>43.7</td><td>53.0</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>38.2</td><td>50.2</td><td>67.7</td><td>76.1</td><td>36.0</td><td>46.9</td><td>30.5</td><td>36.6</td><td>43.1</td><td>52.4</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>40.1</td><td>50.9</td><td>68.1</td><td>75.6</td><td>30.8</td><td>40.2</td><td>26.5</td><td>31.9</td><td>41.4</td><td>49.7</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>35.2</td><td>47.2</td><td>69.3</td><td>77.2</td><td>35.6</td><td>46.8</td><td>34.7</td><td>40.2</td><td>43.7</td><td>52.8</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td><td>Ext.</td><td>39.5</td><td>51.9</td><td>69.6</td><td>77.8</td><td>38.1</td><td>49.4</td><td>35.4</td><td>41.0</td><td>45.7</td><td>55.1</td>
  </tr>
  <tr>
    <th colspan="12">Top-20 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>42.1</td><td>55.0</td><td>71.1</td><td>79.1</td><td>39.4</td><td>51.1</td><td>35.4</td><td>42.4</td><td>47.0</td><td>56.9</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>37.2</td><td>50.0</td><td>65.6</td><td>74.0</td><td>32.4</td><td>43.2</td><td>30.3</td><td>35.7</td><td>41.4</td><td>50.8</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>37.6</td><td>49.1</td><td>66.4</td><td>74.4</td><td>33.6</td><td>42.7</td><td>25.6</td><td>30.5</td><td>40.8</td><td>49.2</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>36.5</td><td>47.6</td><td>66.6</td><td>74.7</td><td>33.3</td><td>43.3</td><td>30.2</td><td>35.6</td><td>41.6</td><td>50.3</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>38.6</td><td>49.1</td><td>68.4</td><td>75.6</td><td>28.9</td><td>38.0</td><td>24.7</td><td>29.9</td><td>40.1</td><td>48.2</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>37.0</td><td>49.1</td><td>70.5</td><td>78.5</td><td>37.9</td><td>49.4</td><td>35.4</td><td>41.1</td><td>45.2</td><td>54.6</td>
  </tr>
  <tr>
    <td>EXIT (Ours)</td><td>Ext.</td><td>42.5</td><td>55.3</td><td>71.0</td><td>79.0</td><td>39.8</td><td>51.1</td><td>36.5</td><td>42.2</td><td>47.5</td><td>56.9</td>
  </tr>
</table>
Table 23: Token distribution analysis across different Top-k scenarios, models, and datasets. Best results for each dataset are highlighted in bold. Second best results are highlighted in underline.

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="1">NQ</th>
    <th colspan="1">TQA</th>
    <th colspan="1">HQA</th>
    <th colspan="1">2WIKI</th>
    <th colspan="1">AVG.</th>
  </tr>
  <tr>
    <th>#token (%) ↓</th>
    <th>#token (%) ↓</th>
    <th>#token (%) ↓</th>
    <th>#token (%) ↓</th>
    <th>#token (%) ↓</th>
  </tr>
  <tr>
    <td colspan="8"><i>Top-5 Documents</i></td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>-</td>
    <td>723.9 (100.0)</td>
    <td>730.3 (100.0)</td>
    <td>749.2 (100.0)</td>
    <td>784.8 (100.0)</td>
    <td>734.4 (100.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>Abs.</td>
    <td><b>38.0 (5.2)</b></td>
    <td><b>36.9 (5.0)</b></td>
    <td><b>63.3 (8.4)</b></td>
    <td><b>55.1 (7.0)</b></td>
    <td><b>46.0 (6.2)</b></td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>Abs.</td>
    <td>77.5 (10.7)</td>
    <td>79.0 (10.8)</td>
    <td>77.3 (10.3)</td>
    <td>71.4 (9.1)</td>
    <td>78 (10.6)</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>Abs.</td>
    <td>115.6 (16.0)</td>
    <td>103.2 (14.1)</td>
    <td>76.6 (10.2)</td>
    <td>62.1 (7.9)</td>
    <td>98.5 (13.4)</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>Ext.</td>
    <td>43.9 (6.1)</td>
    <td>42.7 (5.8)</td>
    <td>90.2 (12.0)</td>
    <td>97.9 (12.5)</td>
    <td>58.9 (8.0)</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>Ext.</td>
    <td>224.3 (31)</td>
    <td>221.9 (30.4)</td>
    <td>229.2 (30.6)</td>
    <td>234.5 (29.9)</td>
    <td>225.1 (30.7)</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td>
    <td>Ext.</td>
    <td>283.8 (39.2)</td>
    <td>211.3 (28.9)</td>
    <td>190.3 (25.4)</td>
    <td>154.9 (19.7)</td>
    <td>228.4 (31.2)</td>
  </tr>
  <tr>
    <td colspan="8"><i>Top-20 Documents</i></td>
  </tr>
  <tr>
    <td>Original Docs</td>
    <td>-</td>
    <td>2897.4 (100.0)</td>
    <td>2925.2 (100.0)</td>
    <td>2996.6 (100.0)</td>
    <td>3139.7 (100.0)</td>
    <td>2939.8 (100.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td>
    <td>Abs.</td>
    <td><b>26.1 (0.9)</b></td>
    <td><b>38.6 (1.3)</b></td>
    <td><b>64.5 (2.2)</b></td>
    <td><b>51.1 (1.6)</b></td>
    <td><b>43.1 (1.5)</b></td>
  </tr>
  <tr>
    <td>CompAct</td>
    <td>Abs.</td>
    <td>105.4 (3.6)</td>
    <td>102.5 (3.5)</td>
    <td>111.8 (3.7)</td>
    <td>109.5 (3.5)</td>
    <td>106.5 (3.6)</td>
  </tr>
  <tr>
    <td>Refiner</td>
    <td>Abs.</td>
    <td>232.4 (8.0)</td>
    <td>176.0 (6.0)</td>
    <td>117.3 (3.9)</td>
    <td>92.7 (3.0)</td>
    <td>175.2 (6.0)</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td>
    <td>Ext.</td>
    <td>43.4 (1.5)</td>
    <td>40.6 (1.4)</td>
    <td>89.3 (3.0)</td>
    <td>95.7 (3.0)</td>
    <td>57.8 (2.0)</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td>
    <td>Ext.</td>
    <td>550.9 (19.0)</td>
    <td>553.9 (18.9)</td>
    <td>563.3 (18.8)</td>
    <td>595.5 (19.0)</td>
    <td>556 (18.9)</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td>
    <td>Ext.</td>
    <td>1001.2 (34.6)</td>
    <td>635.0 (21.7)</td>
    <td>465.0 (15.5)</td>
    <td>367.1 (11.7)</td>
    <td>700.4 (23.9)</td>
  </tr>
</table>
Table 24: Latency analysis across different Top-\( k \) scenarios, models and datasets. Each entry shows compression/reading/total time in seconds. 8B reader experiments were conducted on a single A100-80GB GPU, while 70B reader experiments utilized 4 A100-80GB GPUs in parallel. Best results for each dataset are highlighted in bold. Second best results are highlighted in underline.

<table>
  <tr>
    <th rowspan="2">Compressor</th>
    <th rowspan="2">Type</th>
    <th colspan="3">NQ</th>
    <th colspan="3">TQA</th>
    <th colspan="3">HQA</th>
    <th colspan="3">2WIKI</th>
    <th colspan="3">AVG.</th>
  </tr>
  <tr>
    <th>Comp.</th><th>Read</th><th>Total</th>
    <th>Comp.</th><th>Read</th><th>Total</th>
    <th>Comp.</th><th>Read</th><th>Total</th>
    <th>Comp.</th><th>Read</th><th>Total</th>
    <th>Comp.</th><th>Read</th><th>Total</th>
  </tr>
  <tr>
    <th colspan="15">Llama-3-1-8B-Instruct</th>
  </tr>
  <tr>
    <th colspan="15">Top-5 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>-</td><td>1.03</td><td>1.03</td><td>-</td><td>0.93</td><td>0.93</td><td>-</td><td>0.96</td><td>0.96</td><td>-</td><td>1.12</td><td>1.12</td><td>-</td><td>1.03</td><td>1.03</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>1.13</td><td>0.43</td><td>1.55</td><td>1.06</td><td>0.29</td><td>1.35</td><td>1.92</td><td>0.32</td><td>2.24</td><td>1.73</td><td>0.38</td><td>2.11</td><td>1.46</td><td>0.36</td><td>1.81</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>8.04</td><td>0.43</td><td>8.47</td><td>8.47</td><td>0.36</td><td>8.83</td><td>7.80</td><td>0.47</td><td>8.26</td><td>7.65</td><td>0.49</td><td>8.14</td><td>7.99</td><td>0.44</td><td>8.43</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>27.40</td><td>0.70</td><td>28.10</td><td>10.50</td><td>0.40</td><td>10.90</td><td>6.50</td><td>0.40</td><td>6.90</td><td>5.80</td><td>0.50</td><td>6.40</td><td>12.52</td><td>0.55</td><td>13.07</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>0.04</td><td>0.50</td><td>0.54</td><td>0.04</td><td>0.31</td><td>0.35</td><td>0.04</td><td>0.37</td><td>0.41</td><td>0.04</td><td>0.59</td><td>0.63</td><td>0.04</td><td>0.44</td><td>0.48</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>0.38</td><td>0.47</td><td>0.85</td><td>0.37</td><td>0.42</td><td>0.79</td><td>0.39</td><td>0.46</td><td>0.84</td><td>0.40</td><td>0.53</td><td>0.93</td><td>0.39</td><td>0.47</td><td>0.86</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td>0.33</td><td>0.43</td><td>0.76</td><td>0.35</td><td>0.36</td><td>0.71</td><td>0.38</td><td>0.40</td><td>0.78</td><td>0.39</td><td>0.53</td><td>0.92</td><td>0.36</td><td>0.43</td><td>0.79</td>
  </tr>
  <tr>
    <th colspan="15">Top-20 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>-</td><td>3.41</td><td>3.41</td><td>-</td><td>2.85</td><td>2.85</td><td>-</td><td>2.94</td><td>2.94</td><td>-</td><td>3.22</td><td>3.22</td><td>-</td><td>3.11</td><td>3.11</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>1.07</td><td>0.63</td><td>1.70</td><td>1.55</td><td>0.31</td><td>1.86</td><td>2.09</td><td>0.35</td><td>2.44</td><td>2.16</td><td>0.44</td><td>2.60</td><td>1.72</td><td>0.43</td><td>2.15</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>25.58</td><td>0.49</td><td>26.06</td><td>24.08</td><td>0.39</td><td>24.47</td><td>27.02</td><td>0.52</td><td>27.53</td><td>31.63</td><td>0.57</td><td>32.19</td><td>27.08</td><td>0.49</td><td>27.57</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>28.00</td><td>0.70</td><td>28.70</td><td>43.70</td><td>0.50</td><td>44.20</td><td>29.00</td><td>0.60</td><td>29.60</td><td>9.80</td><td>1.00</td><td>10.80</td><td>27.63</td><td>0.69</td><td>28.32</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>0.11</td><td>0.51</td><td>0.62</td><td>0.11</td><td>0.28</td><td>0.39</td><td>0.12</td><td>0.42</td><td>0.54</td><td>0.11</td><td>0.56</td><td>0.68</td><td>0.11</td><td>0.44</td><td>0.55</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>1.76</td><td>1.04</td><td>2.80</td><td>1.78</td><td>0.92</td><td>2.70</td><td>1.83</td><td>0.93</td><td>2.76</td><td>1.89</td><td>1.01</td><td>2.90</td><td>1.81</td><td>0.98</td><td>2.79</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td>1.33</td><td>0.42</td><td>1.76</td><td>1.37</td><td>0.36</td><td>1.74</td><td>1.45</td><td>0.40</td><td>1.85</td><td>1.51</td><td>0.53</td><td>2.04</td><td>1.42</td><td>0.43</td><td>1.85</td>
  </tr>
  <tr>
    <th colspan="15">Llama-3-1-70B-Instruct</th>
  </tr>
  <tr>
    <th colspan="15">Top-5 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>-</td><td>8.63</td><td>8.63</td><td>-</td><td>7.70</td><td>7.70</td><td>-</td><td>8.30</td><td>8.30</td><td>-</td><td>9.09</td><td>9.09</td><td>-</td><td>8.43</td><td>8.43</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>1.28</td><td>3.20</td><td>4.48</td><td>1.20</td><td>2.14</td><td>3.34</td><td>2.06</td><td>2.37</td><td>4.43</td><td>1.71</td><td>2.54</td><td>4.24</td><td>1.56</td><td>2.56</td><td>4.12</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>8.77</td><td>3.11</td><td>11.88</td><td>8.97</td><td>2.72</td><td>11.69</td><td>8.28</td><td>2.73</td><td>11.01</td><td>8.36</td><td>3.23</td><td>11.59</td><td>8.59</td><td>2.95</td><td>11.54</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>35.90</td><td>6.60</td><td>42.50</td><td>14.70</td><td>3.60</td><td>18.30</td><td>11.30</td><td>3.30</td><td>14.60</td><td>8.00</td><td>3.20</td><td>11.20</td><td>17.48</td><td>4.16</td><td>21.64</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>0.04</td><td>2.44</td><td>2.48</td><td>0.04</td><td>2.21</td><td>2.25</td><td>0.04</td><td>2.87</td><td>2.91</td><td>0.05</td><td>3.29</td><td>3.33</td><td>0.04</td><td>2.70</td><td>2.74</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>0.50</td><td>3.87</td><td>4.37</td><td>0.50</td><td>3.40</td><td>3.90</td><td>0.51</td><td>3.52</td><td>4.03</td><td>0.54</td><td>3.72</td><td>4.26</td><td>0.51</td><td>3.63</td><td>4.14</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td>0.44</td><td>3.50</td><td>3.94</td><td>0.44</td><td>2.66</td><td>3.10</td><td>0.42</td><td>2.88</td><td>3.30</td><td>0.50</td><td>3.03</td><td>3.54</td><td>0.45</td><td>3.02</td><td>3.47</td>
  </tr>
  <tr>
    <th colspan="15">Top-20 Documents</th>
  </tr>
  <tr>
    <td>Original Docs</td><td>-</td><td>-</td><td>25.78</td><td>25.78</td><td>-</td><td>24.85</td><td>24.85</td><td>-</td><td>25.35</td><td>25.35</td><td>-</td><td>28.09</td><td>28.09</td><td>-</td><td>26.02</td><td>26.02</td>
  </tr>
  <tr>
    <td>RECOMP-Abst</td><td>Abs.</td><td>1.13</td><td>3.36</td><td>4.49</td><td>1.58</td><td>2.11</td><td>3.70</td><td>2.24</td><td>2.59</td><td>4.83</td><td>2.07</td><td>2.75</td><td>4.81</td><td>1.76</td><td>2.70</td><td>4.46</td>
  </tr>
  <tr>
    <td>CompAct</td><td>Abs.</td><td>27.60</td><td>3.24</td><td>30.84</td><td>25.41</td><td>2.74</td><td>28.15</td><td>28.90</td><td>3.08</td><td>31.99</td><td>33.54</td><td>2.92</td><td>36.45</td><td>28.86</td><td>2.99</td><td>31.86</td>
  </tr>
  <tr>
    <td>Refiner</td><td>Abs.</td><td>32.50</td><td>5.40</td><td>37.90</td><td>47.40</td><td>1.50</td><td>48.90</td><td>31.40</td><td>1.50</td><td>33.00</td><td>9.70</td><td>3.10</td><td>12.80</td><td>30.25</td><td>2.90</td><td>33.15</td>
  </tr>
  <tr>
    <td>RECOMP-Extr</td><td>Ext.</td><td>0.11</td><td>2.27</td><td>2.38</td><td>0.12</td><td>2.10</td><td>2.21</td><td>0.12</td><td>2.77</td><td>2.89</td><td>0.13</td><td>3.18</td><td>3.31</td><td>0.12</td><td>2.58</td><td>2.70</td>
  </tr>
  <tr>
    <td>LongLLMLingua</td><td>Ext.</td><td>2.35</td><td>8.51</td><td>10.86</td><td>2.35</td><td>8.04</td><td>10.38</td><td>2.40</td><td>8.24</td><td>10.65</td><td>2.50</td><td>8.47</td><td>10.97</td><td>2.40</td><td>8.32</td><td>10.71</td>
  </tr>
  <tr>
    <td>Ours (EXIT)</td><td>Ext.</td><td>1.62</td><td>3.50</td><td>5.12</td><td>1.64</td><td>2.67</td><td>4.31</td><td>1.78</td><td>2.88</td><td>4.65</td><td>1.80</td><td>2.96</td><td>4.75</td><td>1.71</td><td>3.00</td><td>4.71</td>
  </tr>
</table>

Reviewer paper summaries: This paper proposes EXIT, an extractive compression method designed to effectively and efficiently compress retrieved contexts within Retrieval-Augmented Generation (RAG) systems. EXIT segments retrieved documents into sentences and performs binary classification based on the entire document context and query. Then, sentences exceeding a threshold are selected and reconstructed in their original order. In results, EXIT demonstrates improved accuracy and reduced inference latency compared to existing compression methods in both single-hop and multi-hop tasks.

This paper introduces EXIT, an extractive compression method tailored for improving the efficiency and accuracy of RAG systems. EXIT operates by segmenting retrieved documents into sentences and using a binary classifier to determine their relevance in the context of the entire document and query. The selected sentences are then reconstructed in their original order to generate a compressed yet informative context. Experimental results demonstrate that EXIT enhances performance in both single-hop and multi-hop QA tasks while significantly reducing inference latency and token overhead.
Summary of strengths: - The method utilizes extractive compression to overcome latency and computational issues associated with existing abstractive compression methods.
- By dynamically selecting sentences based on the complete retrieved context and query, the method is adaptive to various tasks.
- The suggested method can be easily integrated into existing RAG pipelines, enhancing scalability.
- It maintains or improves performance even when the number of retrieved documents increases (i.e., when top-k becomes large).

1. The proposed method named EXIT effectively balances efficiency and accuracy by leveraging extractive compression, mitigating the latency issues of abstractive approaches.
2. The framework dynamically selects relevant sentences in a single-step process, making it adaptable to diverse query settings.
3. Empirical results highlight that EXIT maintains or even improves performance as the number of retrieved documents increases, ensuring scalability.
Summary of weaknesses: - As the classifier is trained solely on HotpotQA data, its generalizability to other domains is limited. Performance may degrade in specialized areas, such as science or biomedical domains, if essential information is omitted.
- Since binary classification evaluates relevance at the sentence level, if a sentence is very long and contains many unnecessary details, the approach could have disadvantages compared to abstractive compression methods. Specifically, it may inadvertently include irrelevant information or noise, reducing the clarity and efficiency of extracting only the core information required to answer the query.
- Reconstructing documents by simply retaining their original sentence order may lead to unnatural context or ambiguity, particularly if many sentences are removed and the remaining ones are insufficient to maintain the original coherence.
- The performance of the classifier was validated using classification metrics such as ROC-AUC, Precision, Recall, and F1. However, this evaluation was primarily limited to the HotpotQA dataset, indicating the need for additional verification to confirm whether the classifier maintains its high performance in other domains.
- Additionally, the classifier’s performance should be compared with that of a frozen model (e.g., Gemma) to verify whether training the classifier indeed results in substantial improvements.

The reviewer has several concerns.

1. Does sentence-level classifier truly reflect the overall documents? The EXIT framework relies on explicit sentence-level annotations for training its classifier. Therefore, as also authors mentioned, current annotations are manual and dependent on HotpotQA data with limited scalability. Thus, the reviewer concerns that the sentence-level annotation, which seems to be limited in reviewer's perspective, truly identify and classify the principal sentences in the overall retrieved documents.

2. After processing the compression and decide the relevant sentence in Figure 2, the unnatural sentence-text could be aligned by appending after query. The reviewer thinks that reconstructing compressed documents in their original sentence order may result in unnatural or ambiguous contexts, especially when a substantial portion of sentences is removed. Does the authors consider these irregular situations additionally? 

Minor additional question.
- The binary classification seems effective and efficient, however the method could handle the noisy information (long context of specific sentence, ambiguous entity) which could be retrieved. The reviewer could conjecture that this problem is not directly related to the compression method. However, there could be some situations that noisy information of specific sentence could be exist and how these compression method (i.e., EXIT) could address these.
Comments, suggestions and typos: - Were there any instances identified or analyzed in which removing multiple intermediate sentences resulted in unnatural or incoherent reconstructions, despite preserving the original sentence order?
- Could generalization have been further improved by generating and validating additional training data using large language models (LLMs), beyond the HotpotQA dataset?
- Is there any reason to specifically selecting Gemma as the classifier? Additionally, why was an encoder-based model not considered?
- A similar dataset, MuSiQue(Trivedi et al) which has questions with more hops(3 hops, 4 hops), also exists. Is there any reason for excluding this dataset from the evaluation?

- The authors depict the compression model as only training part in the overall figure. However, the frozen retrieved & reader models are not fully compared.
Overall assessment scores: 3

3
Metareview score: 3
Metareview: The paper proposes EXIT, an extractive context compression framework that enhances both the effectiveness and efficiency of RAG systems. Unlike prior extractive methods that treat sentences independently, EXIT adopts a context-aware classifier that evaluates sentence relevance while preserving document-level context, enabling efficient and adaptive compression tailored to query complexity and retrieval quality.

Overall, reviewers found the paper well-written and the proposed method practically useful, with strong empirical results across different datasets. However, some concerns were raised. Reviewer T4bx questioned the simplicity of the binary classification model, particularly may not work for more complex or long-document scenarios, and pointed out modest compression rates. Reviewer ubGL also raised issues about the unnatural sentence-text and the scalability of supervised sentence-level annotation.
Summary of reasons to publish: 1. The proposed EXIT method introduces a context-aware sentence classifier that improves over prior extractive methods by preserving document context and enabling dynamic compression.
2. Extensive empirical experiments across various datasets valid the effectiveness and efficiency of the proposed EXIT method.