COMI: Coarse-to-fine Context Compression via Marginal Information Gain
Authors: Jiwei Tang
Shilei Liu
Zhicheng Zhang
Yujin Yuan
Libin Zheng
Wenbo Su
Bo Zheng
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Long Context
Primary area: foundation or frontier models, including LLMs
TL;DR: Coarse-to-fine Context Compression via Marginal Information Gain

Abstract: Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse tasks. However, their deployment in long context scenarios remains hindered by computational inefficiency and information redundancy. Context compression methods address these challenges by significantly reducing input length and eliminating redundancy. We propose COMI, a coarse-to-fine adaptive context compression framework that jointly optimizes for semantic relevance and diversity under high compression rates. We introduce Marginal Information Gain (MIG), a metric defined as the relevance of a unit to the input query minus its semantic redundancy with other units, guiding the compression process to prioritize information that is both relevant and low redundant. The framework operates in two stages: (1) Coarse-Grained Group Reallocation, where the context is partitioned into groups and dynamically assigned compression rates based on inter-group MIG, ensuring compression budgets align with information value distribution; and (2) Fine-Grained Token Merging, where tokens within each group are fused via an intra-group MIG-based weighting mechanism, thereby preserving key semantics while avoiding the accumulation of redundancy. Extensive experiments across question-answering (e.g., NaturalQuestions, 2WikiMQA, HotpotQA and NarrativeQA), summarization (e.g., MultiNews) with various backbones (e.g., LLaMA-2-7B, Qwen2-7B) show that COMI outperforms existing baselines by a large margin, e.g., approximately 25-point Exact Match (EM) improvement under 32x compression constraint with Qwen2-7B on NaturalQuestions

Published as a conference paper at ICLR 2026
COMI: COARSE-TO-FINE CONTEXT COMPRESSION VIA MARGINAL INFORMATION GAIN
Jiwei Tang Shilei Liu
Zhicheng Zhang'
Yujin Yuan?
Libin Zheng Wenbo Su?
Bo Zheng** 2+
TTsinghua University Future Living Lab of Alibaba SSnn Yat-sen University
tangjw246maills testtsnnggn innguu.. edu. cn zheng1b6@mail sysu. edu. cn
bozhengealibaba inc. com
ABSTRACT
Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse tasks. However, their deployment in long context scenarios re- mains hindered by computational inefficiency and information redundancy. Con- compression methods address these challenges by significantly reducing input length and eliminating redundancy. We propose COMI, a coarse-to-fine adaptive context compression framework that jointly optimizes for semantic relevance and diversity under high compression rates. We introduce Marginal Information Gain (MIG), a metric defined as the relevance semantic redundancy with other units, guiding the compression process to prior- itize information that is both relevant and low redundant. The framework oper- ates in two stages: (I) Coarse-Grained Group Reallocation, where the context is partitioned into groups and dynamically assigned compression rates based on inter- group MIG, ensuring compression budgets align with information value dis- tribution; and (2) Fine-Grained Token Merging, where tokens within each group are fused via intra-group MIG-based weighting mechanism, thereby preserving key semantics while avoiding the accumulation of redundancy. Extensive experi- ments across question-answee answering (e.g., NaturalQQestions, 2WikiMQA, HotpoiQA and NarrativeQA), summarization (e.g.. MultiNews) with various backbones (e.g.. LLaMA-2-7B, Qwen2-7B) show that COMI outperforms existing baselines by a large margin, e.g., approximately 25- 25-point Exact Match (EM) improvement under 32x compression constraint with Qwen2-7B on NaturalQuestion!]
unit to the input query minus its
INTRODUCTION
Large Language Models (LLMs) achieve exceptional performance across a wide range of Natural
Language Processing (NLP) tasks (Yang et al., 2024 DeepSeek-Al et al. 2025 Team et al. 2025
Lv et al. 2025 Zhao et al. 2025b; Liu et al. (025a). However, recent advances in prompting
techniques, such as Retrieval- Auggeelt---emeeeedddee Generation (RAG) (Lewis et al. 2020), inevitably increase input length, introducing two key challenges when deploying LLMs in long context scenarios: (1) computational cost, as the quadratic complexity of the attention mechanism in Transformer (Vaswani al.
2017) leads to inefficiency with long sequences; and (2) information redundancy, where the presence of redundant content degrades model performance (Jiang et al. 2024 Ge et al. 2024
al. 2024b Tang al. 2025a b).
Context compression emerges as a promising solution in the NLP community to address these chal- lenges by significantly reducing input length and eliminating redundancy. Existing prompt com- pression methods mainly fall into two categories: task agnostic context compression methods (Xu
2024 Pan al.] 2024) Ge al.) 2024 Tan et al. 2024 Cheng et al. 2024 Zhang et al.]
2025
Ye et al.) 2025 Li et al. [025] Liao et al. 2025| Tang et al. (025b) and task- context
compression methods (Cao et 2024 Jiang et al.] 2024 Zhao et al.] 2025c Tang al.) 2025a
Fang et al. 2025 Liskavets al. 2025
LLMs typically allocate attention to only small subset of
"Corresponding authors
'The code will be available wahhtpss//githhii http:///tttuub.coom///i////g////

---

Published as a conference paper at ICLR 2026
query-relevant tokens (see Figure lap. Consequently, task-agnostic methods that compress context without considering the input query mevitably lose or dilute relevant information, especially at high compression rates.
Total Attention Contribution (%6
39
0.05
84
0.75
0.65
0.25
1.00
1.25
1.75
2.00
Top Percentage of Query-Relade Rellated Takens 1%)
Taken Index
(a) Attention Weight Contribution.
(b) Top Query-Related Tokens Similarity.
Figure 1: Analysis of Attention Distribution and Similarity of Top Query-Relater Tokens. (a) Only small number of tokens related to the query occupy a large proportion of the attention weight allocation; for example, These query-related tokens are highly similar to each other, with the lowest similarity exceeding 0.6. In contrast, task-aware methods generally include query in the compression process, preserving information according relevance via query-guie suery-guided strategies such as merging, deletion sum- marization. However, existing task-aware methods rely solely on relevance as criterion for com- pression, ignoring the inherent redundancy natural language (Shannon) 1948). This leads to the retention of highly similar relevant content, with high redundancy (see Figure 1b Such over- similarity can mislead the LLM into producing erroneous outputs (relevance does not guarantee cor- rectness) (Yang al. 2025 Wang & Sun 2025). This issue is especially amplified in long-context scenarios, where different segments carry varying information value and should compressed with different compression rates (Jiang et al. 2023 2024 Tang al. 2025a). Existing dynamic com- pression rate allocation mechanisms are limited in several ways: they either follow predefined linear rules lacking adaptability (Jiang et al.) 2023 2024 Tang et al. (025a), rely on model-internal under- standing (Chen et al., 2025) (i.e., task-agnostic that may ignore query-relevant content, or determine compression rates based solely on relevance (Cao al. 2024) (i.e., assigning lower compression to high relevance content). None of these methods account for semantic redundancy among compres- sion units, leading to repeated retention of similar information and compromising both compression effectiveness and information diversity. This naturally raises a research question: How can we retain query-relevant information while identifying and eliminating semantic redundancy among compressed representations, especially under high compression rales, to jointly optimize relevance and diversity?
0.75% most relevant tokens occupy 99% of the attention weights. (b)
To this end, we propose COMI (COarse-to-fine foniee Context Compression via Marginal Information Gain), a coarse-to-fing context compression framework that adaptively balances relevance informa- tion preservation and redundancy climination. We introduce Marginal Information Gain (MIG), which defined as the relevance of a unit to the query minus its semantic redundancy with other units. This metric jointly captures relevance and semantic uniqueness, guiding the compression process to prioritize information that is both relevant and low redundant. COMI employs coarse- to-fine compression strategy. In the first stage, Coarse-grained Group Reallocation, the input context is divided into segments of equal length, each treated as a compression group. MIG is com- puted for mnte-grroup and the compression rate for each group is dynamically adjusted accordingly: groups with high MIG (i.e., high relevance and low redundancy) are assigned lower compression rates. This enables the compression budget to be adaptively reallocated based on the distribution of information value in the context. In the second stage, Fine-Grained Token Merging, tokens are weighted by their intra-group MIG, and fine-grained semantic fusion is performed accordingly. To- kens with high MIG contribute more to the merged representation, ensuring that key semantic units are preserved while avoiding the accumulation of "relevant but redundant" content. Through this hierarchical compression mechanism, COMI effectively retains high-relevance, low-redundancy in- formation even under high compression rates, ensuring that the final compressed representation is semantically complementary rather than redundancy, thereby increasing information diversity.
2

---

Published as a conference paper at ICLR 2026
Our contributions three-fold: (1) We introduce Marginal Information Gain (MIG) metric for context compression, jointly modeling task relevance and semantic redumdancy. This overcomes the limitations of existing relevance-only methods and provides more discriminative framework for evaluating information value in long context compression. (2) We propose COMI, which em- ploys coarse-to-fine adaptive compression strategy. At the coarse level, reallocation based on inter-group MIG dynamically adjusts compression rates across different regions. At the fine level, intra-group MIG-guided weighted fusion eliminates group redundancy, preventing the accumula- tion of similar content. (3) We conduct comprehensive experiments on long-con cong-context tasks including muestion-Answeree (QA), summarization. Experimental results demonstrate that COMI outper- forms existing methods by large margin under high compression rates, e.g., with a 32x compres- sion constraint and using Qwen2-7B as the backbone, COMI improves the Exact Match (EM) score by approximately 25-point over suboptimal baseline on NaturalQuestions.
PRELIMINARY: GMSA
GMSA Tang et al. 2025b) introduces an issue of cross-layer semantic misalignment in the encoder- decoder based framework and addresses it via Layer Semantic Alignment (LSA), which aligns high- level summary vectors with low-level original input semantics, thereby bridging the semantic gap across different layers. Since COMI also relies on an encoder-decoder framework, we similarly em- ploy LSA to achieve cross-layer semantic alignment (see Figure 2 Following Tang et al. 2025b). we set LSA single layer.
3 RELATED WORK
Task-Agnosti Context Compression Methods. Task-agnostic context compression methods typically do not incorporate the input query and aim to preserve overall semantic information for broad downstream applicability. Main approaches include: (1) Encoder-decoder methods (Ge et al.,
2024 Cheng et al., [2244] Tan et al. 2024 Liao et al. 2025 Li al. 2025 Rau et al. 2024 Dai
et al. 2025 Choi et al. 2025 Tang et al. 20256 Zhao et al. 2025a Liu et al. 2025b). which
compress input via an encoder and decode answers from the compact representation; (2) Attention
mask modification (Mu et al. 2023 Petrov et al.
2025 Ye et al., (025), learning compact soft tokens guided by task losses; (3) Autoregressive modeling (Chevalier et al. 2023 Zhang et al,,
2025), treating compression as sequential generation process conditioned on prior compressed segments; (4) Metric-driven methods using entropy (Li et al. [2033] Jiang et al. (203) or bidirec- tional semanties (Pan al. 2024) to remove less important content; and (5) Summarization-baat methods (Xu et al.) 2024). training a task-agnostic summarizer for compression. Despite preserv- ing general semantics, these methods lack awareness of query-related, tokens, inevitably discarding useful information and degrading performance, especially under high compression constraints.
Task-Aware Context Compression Methods. Task-aware context compression methods do not pursue the completeness of semantic over the compressed representation, but focus on retaining information relevant to downstream task. During the compression process, these methods simul- taneously receive the original context and a query to merge relevant content (Cao al.. 2024).
summarize (Yoon et al. 2024; Hwang et al. (202)) or delete irrelevant content (Jiang et al. 2024 Tang et al.] 2025a]] Liskavets et al. 2025) Fang et al., 2025 Zhao et al.) (025c). Although these methods can filter task-relevant content, they implicitly assume conditional independence among the retained tokens, leading to significant redundancy within the preserved tokens and making it easy to mislead LLM into generating erroneous outputs.
KV-ccche Compression. KV-cache Compression methods compresses KV-caches layer-wise, ex- ploring strategies like lnter-layer sharing (Brandon et al.) 2024). reducing heads (Shazeer 2019 Ainslie et al. 2023), discarding less important KVs (Xiao al., 2024 Li et al. 2024 Zhang et al. (025)). The Key limitations on these methods include requiring identical compression and response models, and necessitating model-specific modifications for different KV-cache structures.
3

---

Published as a conference paper at ICLR 2026
Contect Takens
Query Tokens
Compressed Tokens
Trainatiee
Presen
Outbut
Decoder
Self-Actn
Decoder
Cill
Doooder Tokortzer
mare
Be-1
Lavar Nomiik
LSA
Quection
Woo
Wo 5
We
Encoderá
Layer Nomm
Input
Organal Corter
X
Oundton
Figure 2: The Training Paradigm of COMI. COMI is based on an encoder-decoder architecture. The original context X and query Q first encoded into hidden states, which then compressed via a compression process (see Figure The compressed representation decoded and trained using cross-entropy loss. During training, the encoder and LSA are fully fine-tuned, while the decoder is partially fine-tuned, updating only the Wo. WK. Wv, and Wo matrices in each layer.
Redundancy
Relevanee
Tokena Group
Reprasentstive Token
L Average Pooling Tokena
Group Reallocation
00
Fine-Graino Group
MI
A/4
Figure 3: The Compression Process of COMI. Specifically, it sequentially performs three steps: I. Average Pooling of Query Tokens. Obtain a single query vector via average pooling; II. Coarse- Grained Group Reallocation. Reallocate the sizes of compression groups based on inter-group Marginal Information Gain (MIG) (i.e., groups with higher MIG are assigned lower compression rates); III. Fine-Grained Token Merging, Compute the intra-group MIG for each token and merge all tokens within a group into a single compressed token according to weights wi...., #L,-1-
4 COMI
In this section, elaborate COMI, coarse-to course-to-fine context compression framework based on incoder-decoder architecture. COMI simultaneously models the relevance of each compression unit the input question and the redundancy among units via the Marginal Information Gain (MIG). then performs Coarse-grained Group Reallocation followed by Fine-grained Group Token Merging retain information that both relevant and low redundant.
4.1 MARGINAL INFORMATION GAIN
For given token 2; query vector q. and the context X to which x; belongs, compute its Marginal Information Gain (MIG) G( q, X) as follows:
I Ii
G( 4, X)
max
(1)
lll
2,
EXx
||05|| ||
4

---

Published as a conference paper at ICLR 2026
Here, the first term measures the cosine similarity between 2; and the query q, representing its rele- vance; the second term captures the maximum cosine similarity between z; and any other token in X, reflecting its redundancy. We demonstrate that using MIG yields superior expected performance compared to relying solely on relevance. (See Appendix A
4.2
COARSE-GRAINES GROUP REALLOCATION
For a given original context X, we first encode via a language model (i.e., encoder) to extract semantic information.
H = EEncoder(X),
(2)
where H is the last hidden state.
Then, divide H into m equal-length, non-overlapping segments // (8 S2, ..,..., Sm.. Given an input query Q = {11: 92...... 9n J, different segments exhibit varying degrees of relevance to Q as well as differing levels of internal redundancy. Therefore, fixed compression rate across all segments is suboptimal. Instead, we propose dynamic compression rate reallocation strategy based on marginal information gain.
First, we average pooling the query sequence Q into single query vector
q =
Σ
9k.
(3)
Q
TeQ
Then we select the representative vector h2 of S; that exhibits the highest relevance to the query via:
h 7
h4 = argmax h, ES;
(4)
[1/ ||||||
Then, we can compute the marginal information gain G(,,, q. H) for each segment S; as defined in Equation (I), which now evaluates the trade-off between the segment's relevance to the query and its redundancy with neighboring segments.
Since segments with higher marginal gain are more informative and less redundant, they should be preserved more faithfully, ie., assigned a smaller compression rate. achieve this, we apply an inverse transformation to reverse the MIG ranking. We cant get the allocation weights via:
G(h,,
P.
=
(5)
j=1 e G(h...8)
where P:-P: 11 These weights P. determine the proportion of the total allowed output length allocated to each segment.
Finally, the target length (i.e., number of tokens) for the compressed representation of segment S,, denoted L,, is computed as:
L Toury P::
(6)
where Lang is the length of the original input sequence.
This dynamic reallocation mechanism enables COMI to efficiently allocate compression resources based on semantic importance and redundancy, preserving segments with higher marginal gain adap- tively and improving downstream task performance under limited context length.
4.3 TINE-GRAINER TOKEN MERGING
After dynamically reallocating the target lengths L; for each compression segment St, proceed to perform token merging within each segment to achieve compression. The goal to preserve maximal informative content with respect to the query while minimizing redundancy among tokens.
For each segment S; = fh,,hh.......................... h2...., his; J. we compute the marginal information gain G(h,, 5,) for every token 2g S; using the same formulation as in Equation (1. This score reflects trade-off

---

Published as a conference paper at ICLR 2026
between the token's ss relevance to the query (via cosine similarity with @) and its redundancy with the most similar token within the same segment.
To merge all tokens in S into a single output token h; that preserves the most informative content, perform MIG-we weightett merging via:
((hh.. h& Emnes, eG(hx.....)
h4
RyES;
That is, each token in the corresponding group is weighted by the softmax of its MIG, ensuring that tokens with higher information gain contribute to the merged compressed representation h;- We can get the total compressed representation X by:
x= {h1 @ h2 @ ... 3 hm):
(8)
where @ denotes concatenation.
4.4 TRAINING OBJECTIVE
We fine-tune COMI following the joint instruction-tuning approach (Lin et al. 2024), enabling to generate correct outputs given query and the original context. The difference lies in that perform fine-tuning based on compressed representations, simultaneously training the encoder, LSA (to ensure the effectiveness of the compressed representations), and the decoder's WQ. WK. Wv. and Wo (to ensure knowledge extraction capability of the Decoder) (see Figure 2
Cull = Σ log Ps (a: LSA (X), 41, 2....., In azii)
(9) where La refers the length of ground truth; LSA (-) denotes the LSA module; po(-) is the Decoder probability distribution obtained after the softmax function, and denotes the i-th token in the predicted answer.
EXPERIMENTS
NaturalQuestions
2WikiMQA
COM1120
70
(DP) CON (FD
COR1 (T2)
60
50
Score
Score
45
20
18
Compression Constraint
Compression Constraint
(a) Pressure test on NaturalQuestions.
(b) Pressure 2WikiMQA.
Figure 4: Compression Pressure Test on NaturalQuestions and 2WikiMQA. As the compression constraint increases, although both COMI and Activation Beacon generally show a downward trend in EM and F1, COMI consistently remains higher than Activation Beacon.
In this section, we seek to answer the following four research questions: (1) How does COMI perform various tasks? (RQ1) (2) What is the impact of compression constraint on COMI's performance? (RQ2) (3) How effective is each individual component within COMI? (RQ3) (4) How does COMI impact native long context LLMs?

---

Published as a conference paper at ICLR 2026
Table 1: Experimental results on four QA benchmarks and the MultiNews summarization dataset. We bold the optimal and underline the suboptimal of baselines. EM refers to Exact Match and F1 refers the FI score. Closed-book indicates using only the input question the input, while Original Prompt indicates using all retrieved documents the input.
Nattraluuestions
2WIkIMQA
HotpotQA
NarrativeQA
MultiNews
Methods
EM
F1
EM
F1
EM
F1
EM
F1
F1
LLaMA-2-7B-CHA
Closed-book
20.21 15.04
21.89
24.92 30.82
27.81 37.80
17.22 34.34
24.03
1.2
10.78
Original Prompt
26.74
47.50
2.16
13.56
33.2
Owen2-7B-Inttee
Closed-book
22.64 72.35
20.15
30.07
20.97
2.82
10.18
2.82
10.18
Original Prompt
38.01
59.78
47.52
64.17
53.68
24.53
9.74
31.42
16x compression constraint
LLaMA-2-7B-Chat
StreamLLM
8.06
18.72
15.92
14.45
24.66
SnapKV
11.90
26.49
25.82
29.55
26.09
37.87
19.79
Activation Beacon
4.56
16.58
2.20
13.84
11.41
24.68
6.39
21.16
26.36
ICAE
1.50
5.93
12.92
21.52
8.24
16,90
0.00
3.86
20.85
LMLinnM
11.64
23.12
28.42
32,05
24.75
35.22
2.44
15.91
20.66
JLMLingua-2-lee
17.74
27.91
24.26
26.60
15.71
25.95
4.51
17.38
20.83
GMSA
10.36
15.63
22.68
28.46
16.09
25.43
1.13
9.60
21.10
COMI
22.75
27.32
41.43
48.74
36.60
49,50
20.32
27.30
Qwen2-7B-Instruce
SnapKV
6.06
19.64
8.09
25.52
13.61
30.84
0.00
11.12
25.56
Activation Beacon
11.67
27.53
49.11
39.73
55.25
33.60
LongL_LMinguaa
15.07
29.64
21.80
29.63
21.58
35.36
11.99
23.16
JLMLingua-2-lart
10.36
20.31
18.26
25.35
2.44
11.28
8.27
11.28
21.39
GMSA
24.41
28.27
29.49
34.23
21.54
30.74
1.88
10.94
26.55
COMI
56.31
55,52
52.13
60.43
45.11
60.52
13.25
24.69
35.85
32x compression constraint
LLaMA-2-7B-Chat
StreamLLM
4.10
11.86
9.76
12.07
8.11
12.97
0.00
0.00
19.28
SnapKV
9,91
24.76
1.53
2.98
8.25
13.30
0.00
0.00
14.44
Activation Beacon
1.47
11.76
1.71
13.38
13.94
26.75
4.79
17.02
24.35
ICAE
1.31
10.82
12.91
21.68
8.20
16.95
0.00
3.16
19.00
LongLLMMingua
7.50
18.86
28.32
31.76
25.12
34.78
2.44
15.10
18.56
LLML.inuaa-2-larr
14.09
22.76
23.89
14.02
23.33
3.57
15.50
16.46
GMSA
9.53
13.94
24.83
28.62
27.15
1.13
8.56
19.60
COMI
16.99
23.94
37.34
43.85
31.46
43.32
6.83
15,90
24.71
Qwen2-7B-Instrue
SnapKV
5.76
17.93
7.16
21.36
10.57
25.70
0.00
10.98
21.35
Activation Beacon
8.52
22.26
38.79
49.91
37.77
$2.76
5.18
21.29
32.38
LongLLMMingua
13.37
27.39
21.19
28.91
22.59
35.45
1.03
11.91
21.60
LLMiingua-2llarg
9.60
18.16
18.81
24.90
2.26
10.27
2.26
10.27
17.70
GMSA
24.63
27.81
28.59
32.81
21.28
30.08
2.73
11.04
25.49
COMI
49.15
49,59
48.89
57.07
40.46
55.24
11.18
22.30
33.60
5.1 SETTINGS
Training. COMI requires only a single training run to be effectively applied multiple downstream tasks (i.e. QA, summarization). We sampled 20,000 examples each from NaturalQuestions (Liu et al. 2024a). HotpoiQA (Yang al. 2018), 2WikiMQA (Ho et al. 2020), NarrativeQA (Kociský et al. 2018). and MultiNews (Fabbri et al. 2019) to form the final training set. All training and testing samples have token lengths no longer than 32K. During training, the batch size is set to 64, the learning rate is set to le-5, and a linear decay schedule is employed. The training paradigm illustrated in Figure 2 During training, randomly sample compression rate (e.g., 16x or

---

Published as a conference paper at ICLR 2026
32x) for each training sample. To ensure fair comparison, train GMSA in the same datasets and settings.
Implementation. Our implementation is based LLaMA-2-7B (Chat) and Qwen2-7B (Instruct). To ensure a fair comparison, all baseline results are re-implemented using official open-source code. All experiments conducted using the Hugging Face framework 8 NVIDIA H20 (94GB) GPUs.
Evaluation Metries. For question answering (i.e., NaturalQuestions, HotpotQA, 2WikiMQA, and
NarrativeQA), we report both Exact Match (EM) (Lewis et al. 2020) and F1 score (Yang et al. 2018 Summarization performance on MultiNews is measured by FI score.
Baselines. We conduct comprehensive comparisons with various methods in both context compres- sion and KV-cache compression, including hard prompt compression methods (i.e, LongI.LML.i gua (Jiang et al. 2024), LLMLingua-2-lart
(Pan et al. (0244), soft prompl compression methods (i.e., ICAE (Ge et al. (022)), GMSA (Tang et al. (025b) ), and KV-cache compression methods (i.e.
StreamLLM (Xiao al. 2024). SnapKV (Li et 2024
Activation Beacon (Zhang et 2025
5.2 MAIN RESULT
For ROI, COMI demonstrates superior performance on both question answering (QA) and summa- rization tasks (see Table [T] On QA tasks, where input questions are typically specific and relate to a particular piece of information within a long context, COMI nearly achieves state-of-the-art results across all compression constraints and evaluation metrics. This holds true for single-hop questions (i.e., NaturalQuestions)) multi-hop questions (i.e., HotpotQA, 2WikiMQA), and QA on extremely long texts (i.e., NarrativeQA, which we test samples with maximum length of 32K). For instance, with a compression rate of 32x and using Qwen2-7B ---nnrructt as the backbone, COMI improves the Exact Match (EM) score by approximately 25 over suboptimal bascline. This high- lights COMI'S exceptional performance under high compression rates. The EM metric reflects the upper bound of model's answer, while the F1 score additionally requires the model's 's output length to be as close to the ground truth as possible. The strong performance on both metrics indicates that COMI not only answers questions more accurately but also produces outputs that are consis- tent in length with the reference answers. On summarization task, the input query is often global request (e.g., "Please summarize the preceding text"). This setting requires the model to preserve and understand global information. COMI's leading performance on these tasks demonstrates its strong comprehension capabilities even without explicit relevant information, confirming its high robustness.
For RQ2, as shown in Figure 4 with the compression ratio gradually increasing (from 2x 32x), while both COMI and Activation Beacon generally show a downward trend, COMI significantly outperforms Activation Beacon (especially in the EM metric, egg. nearly 40 point higher on the NaturalQuestions dataset under the 32x compression constraint). EM requires the generated an- swer to perfectly match the ground truth, indicating that compared to Activation Beacon, COMI effectively retains key information (directly related to the ground truth) across various compression rates.
5.3 ABLATION STUDY
To address RQ3, we conducted four abla- tion studies to exam- ine how each compo- nent in COMI con- tributes to its perfor- mance (see Table 2):
Table 2: Ablation study analysis on NaturalQuestions, 2WikiMQA under 32x compression constraint using Qwen2-7B.
NaturalQuestions
2WikiMQA
Methods
EM
F1
EM
F1
(1) Ours w/o Coarse- Grained Group Real- location replaces dy- namic grouping with a uniform partition, so every token group
Default
56.31
55.52
52.13
60.43
w/o Coarse-Grained Group Reallocation
54.54
54.26
49.82
58.06
w/o Fine-Grained Tokens Merging
50.81
50.82
47.88
56,45
w/o Yoarse-Grainel level Redundancy
54.59
52.81
50.14
58.64
Fine- Grained level Redundancy
51.53
$2.89
49.50
57.87
8

---

Published as a conference paper at ICLR 2026
is compressed the same rate. (2) Ours wo Fine- Grainer Tokens Merging substitutes our information ceiimattion-gin---i- gain-based weighting with plain average pooling. (3) Ours w/o Gransed level Redundancy reallocates groups based solely on relevance, ignoring inter-group redundancy. (4) Ours w/o Fine-Grained level Redundancy performs token merging without considering redundancy among tokens within the same group.
Removing any component causes clear drop in all metrics, demostrating the necessity and effec- tiveness of each one. Eliminating Coarse-Grained Group Reallocation misallocates the compression budget and risks losing key information, whereas dropping Fine-Grained Token Merging deprives COMI of its fine fine-graing seee-graned group sensitivity and dilutes key details (both harming overall re- sults). Disregarding redundancy at either the coarse-grained or fine- fine-grained stage retains redundant information, weakening the compressed representation, making it harder for the model to learn, and ultimately degrading performance.
5.4 EFFICIENCY ANALYSIS
In this section, analyze the computational efficiency of the COMI. By compressing the original context through coarse-grained group reallocation and fine-grained token merging, COMI signifi- cantly reduces the sequence length processed during generation, thereby lowering inference cost. The overall process consists of two main stages: compression and generation, whose floating-point operations (FLOPs) are modeled separately.
Table 3: Latency Evaluation (seconds) under 32x compression constraint.
Method
Compression
Generation
End-to-End
NarrativeQA
Original Prompt
7.04
SnapKV
2.10
Activation Beacon
3,93
LongLLMLinngu
68.84
2.12
70.95
GMSA
1.57
0.53
2.10
COMI
2.76
0.50
3.27
MultiNews
Original Prompt
8,58
SnapKV
5.03
Activation Beacon
7.94
LongL .MLggnnn
5.44
2.81
8.25
GMSA
0.52
3.34
3.86
COMI
0.66
3.43
4.09
The compression stage consists of two parts: first, the original context partitioned into groups, and each group's size is dynamically adjusted based query relevance and inter- goeer-rrupp redundancy; second, tokens within each group are pooled via weighted to generate compressed representations.
Let Ly the query length, and Lg Lorg/ri /r] the compressed length, where is the compression rate. The FLOPs for the compression stage can be expressed as:
CLoPP.comp GroupItealloo + Prooling (Torgs Ie) + FLSA (Ia),
where Le refers to compressed context length; |GroupRealloc(] denotes the group reallocation cost (complexity O(N, log N, + N2), with Lg & Loogg 2); Fmerging denotes the merging cost (complexity O(r2), with r < Lorg Due to their small input scales, both operations incur only lightweight overhead.
For the generation stage, assuming the answer length is L,, L,, forward passes are required. The FLOPs for the i-th forward pass depend the input sequence length, i.e., T., and query length Tag-

---

Published as a conference paper at ICLR 2026
Thus, the FLOPs for each forward pass given by:
FLOPs of forward
Decoder (Les Lay i).
Combining both compression and generation stages, the total FLOPs is:
FLOPs Σ
FLOPs forward
FLOPs comp
Whether crestion-answering (QA) tasks or generation-centric summarization task, COMI achieves more than 2x end-to-end speedup over the Original Prompt 32x compression straint. For all compression methods, the end-to-end latency can be divided into compression and generation phases (see Table 3 P
In contrast, the Original Prompt incurs only generation latency. THE IMPACT OF COMT ON NATIVE LONG-CONTEXT LLMS
5.5
For RQ4, evaluate the impact of COMI on models with native long-context capabilities, we train COMI using Qwen3-4[B-nnstruct (which natively supports a 256K context length) and use Fl as an evaluation metrie. As shown in Table 4 even compared the strong baseline of feeding the full original prompt, COMI achieves superior performance across all datasets at both 16x and 32x compression constraint. For instance, on the NaturalQuestions dataset at 16x compression, COMI attains an F1 score of 34.79, compared to only 16.90 for the original prompt, demonstrating that COMI can enhance performance even for models with strong native long-context capabilities.
Table 4: The performance of COMI on different QA datasets (Qwen3-4B IB-Ittructt as backbone).
Method
NaturalQuestions
2WikiMQA
HotpotQA
NarrativeQA
16x compression constraint
Original Prompt
16.90
22.51
34.16
11.35
COMI
34.79
45.01
44.44
15.71
32x compression construint
Original Prompt
16.90
22.51
34.16
11.35
COMI
28.89
41.19
39.71
13.25
CONCLUSION
We propose COMI, a coarse-to-fine context compression framework that dynamically optimizes for both task relevance and semantic diversity under high compression rates. By introducing Marginal Information Gain (MIG) (i.e., a metric that penalizes redundancy while rewarding query relevance) COMI adaptively reallocates compression budgets between segments and performs token merging within groups. Extensive experiments demonstrate that COMI significantly outperforms existing methods, e.g., achieving up 25-point EM gain under 32x compression. This work establishes MIG as a critical criterion for efficient and effective long-context modeling in LLMs.
ETHICS STATEMENT
This work introduces COMI, encoder-decoder based framework designed to achieve adaptive coarse-to-fine context compression through marginal information gain. The data and models used in our research are released under open-source licenses and sourced from open platforms. Although our work may have various socictal impacts, it does not introduce any additional ethical concerns compared to existing text compression methods. Therefore, we believe it is unnecessary to specifi- cally highlight any particular ethical issues here.
For SnapKV and Activation Beacon, the latencies of the two phases can not be individually measurable due coupling between compression and generation.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work was supported by Alibaba Group through Alibaba Research Intern Program. Libin Zheng supported by National Natural Science Foundation of China (No. 62472455, U22B2060).
REFERENCES
Joshua Ainslie, James Thorp. Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. GQA: training generalized multi-query transformer models from multi-head head check- points. In EMNLP, pp.
Association for Computational Linguistics, 2023.
Jeff A. Bilmes. Submodularity in machine learning and artificial intelligence. abs/2202.00132, 2022.
CoRR,
William Brandon, Mayank Mishra, Aniruddha Nrusimha, Rameswar Panda, and Jonathan Ragan- Kelley, Reducing transformer key-value cache size with cross-layer attention. In NeurIPS, 2024.
Zhiwei Cao, Qian Cao, Yu Lu, Ningxin Peng, Luyang Huang, Shanbo Cheng, and Jinsong Su. Retaining key information under high compression ratios: Query-guided compressor for Ilms. In ACL (1), pp. 12685-12695. Association for Computational Linguistics, 2024.
Shaoshen Chen, Yangning Li, Zishan Xu, Yongqin Zeng, Shunlong Wu, Xinshuo Hu, Zifei Shan, Xin Su, Jiwei Tang, Yinghui Li, and Hai-Tao Zheng. DAST: context-aware compression in Ilms via dynamic allocation of soft tokens. In ACL (Findings), pp. 20544-20552. Association for Computational Linguistics, 2025.
Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, and Dongyan Zhao. xrag: Extreme context compression for aarrival-auugeener generation with one token. In NeurIPS, 2024.
Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. Adapting language models to compress contexts. In EMNLP, pp. 3829-3846. Association for Computational Linguistics, 2023.
Eunseong Choi, June Park, Hyeri Lee, and Jongwuk Lee. Conflict-awarre soft prompting for retrieval-augmented generation. CoRR, 0as/2508.15253. 2025.
Thomas M. Cover and Joy A. Thomas. Elements of information theory (2. ed.). Wiley, 2006.
Yuhong Dai, Jianxun Lian, Yitian Huang, Wei Zhang, Mingyang Zhou, Mingqi Wu, Xing Xie, and Hao Liao. Pretraining context compressor for large language models with embedding- based memory. ACL (I), pp. 28715-28733 Association for Computational Linguistics, 2025.
Maulik A. Dave. Review of "information theory, inference, and learning algorithms by david j. c.
mackay", cambridge university press, 2003. SIGACT News, 37(4):34-36, 2006.
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. CoRR, abs/2501. 2025.
Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. Multi-news: A large- scale multi-document summarization dataset and abstractive hierarchical model. In ACL (?), pp. 1074-1084. Association for Computational Linguistics, 2019.
Yixiong Fang, Tianran Sun, Yuling Shi, and Xiaodong Gu. Attentionrag: Attention-guided context pruning in retrieval-augmeetee generation. CoRR, abs/2503.10720, 2025.
Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. In ICLR. OpenReview.net, 2024.
James A Hanley and Barbara J McNeil. The meaning and use of the area under a receiver operating characteristic (roc) curve. Radiology, 143(1):29-36, 1982.

---

Published as a conference paper at ICLR 2026
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing A multi- hop QA dataset for comprehensive evaluation of reasoning steps. In COLING, pp. 6609-6625. International Committce on Computational Linguistics, 2020.
Tacho Hwang, Sukmin Cho, Soyeong Jeong, Hoyun Song. Seung Han, and Jong C. Park. EXIT: context-aware extractive compression for enhancing retrieval -augeette generation. In ACL (Findings), pp- 1895-4922 Association for Computational Linguistics, 2025.
Huiqiang Jiang. Qianhui Wu, Chin- Yew Lin, Yuqing Yang, and Lili Qiu, LImlingua: Compress- ing prompts for accelerated inference of large language models. In EMNLP, pp. 13358-13376. Association for Computational Linguisties, 2023.
Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Cein-Yew Lin, Yuqing Yang, and Lili Qiu. LonglImlingua: Accelerating and enhancing Ilms long context scenarios via prompt com- pression. In ACL (1), pp. 1658-1677. Association for Computational Linguisties, 2024.
Tomás Kociský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The narrativeqa reading comprehension challenge. Trans. Assoc. Com- put. Linguistics, 6:317-328, 2018.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In NeurlPS, 2020.
Yucheng Li, Bo Dong. Frank Guerin, and Chenghua Lin. Compressing context to enhance inference efficiency large language models. In EMNLP, pp. 6342-632-3622 6353. Association for Computational Linguistics, 2023.
Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: LLM knows what you are looking for before generation. In NeurlPS, 2024.
Zongqian Li, Yixuan Su, and Nigel Collier. 500xcompressor: Generalized prompt compression for large language models. In ACL(1) (1), pp. 25081-228500 250911. Association for Computational Linguistics, 2025.
Huanxuan Liao, Wen Hu, Yao Xu, Shizhu He, Jun Zhao, and Kang Liu. Beyond hard and soft: Hybrid context compression for balancing local and global information retention. CoRR, abs/2505.15774, 2025.
Xi Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Richard James, Pedro Ro- driguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. RA- DIT: retrieval-augmentte dual instruction tuning. In ICLR. OpenReview.net, 2024.
Barys Liskavets, Maxim Ushakov, Shuvendu Roy, Mark Klibanov, Ali Etemad, and Shane K. Luke. Prompt compression with context-aware sentence encoding for fast and improved LLM inference. In AAAI, pp. 24595-24604, AAAI Press, 2025.
Langming Liu, Shilei Liu, Yujin Yuan, Yizhen Zhang, Bencheng Yan, Zhiyuan Zeng, Zihao Wang, Jiaqi Liu, Di Wang, Wenbo Su, Pengjie Wang, Jian Xu, and Bo Zheng. Uqabench: Evaluating embedding for prompting lims in personalized question answering. In KDD (2), pp. 5652-5661. ACM, 2025a.
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilaequa, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Trans. Assoc. Comput. Linguistics, 12:157-173, 2024a.
Xin Liu, Runsong Zhao, Pengcheng Huang, Xinyu Liu, Junyi Xiao, Chunyang Xiao, Tong Xiao, Shengxiang Gao, Zhengtao Yu, and Jingbo Zhu. Autoencoding-free free context compression for Ilms 2025b.
via contextual semantic anchors. CoRR,
12

---

Published as a conference paper at ICLR 2026
Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, and JingBo Zhu. Forgetting curve: A reliable method for evaluating memorization capability for cong-context models. In EMNLP, pp. 4667-46688 Association for Computational Linguisties, 2024b.
Qingsong Lv, Yangning Li, Zihua Lan, Zishan Xu, Jiwei Tang, Yinghui Li, Wenhao Jiang. Hai-Tao Zheng, and Philip S. Yu. RAISE: reinforenced adaptive instruction selection for large language models. CoRR, abs/2504.07282, 2025.
Jesse Mu, Xiang Li, and Noah D. Goodman. Learning to compress prompts with gist tokens. In NeurlPS, 2023.
George L. Nemhauser, Laurence A. Wolsey, and Marshall L. Fisher. An analysis of approximations for maximizing submodular set functions - I. Math. Program., 14(1):265-294, 1978.
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Vic- tor Rühle, Yuqing Yang, Chin-Yeuu Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. Limlingua- 2: Data distillation for efficient and faithful task-agnostic prompt compression. ACL (Findings), pp. 963-9661111111119 981. Association for Computational Linguistics, 2024.
Hanchuan Peng, Fuhui Long, and Chris H. Q. Ding. Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy.. IEEE Trans. Puttern Anal. Much. Intell., 1266:1226-1133 2005.
Aleksandar Petrov, Mark Sandler, Andrey Zhmoginov, Nolan Miller, and Max Vladymyrov, Long context in-context compression by getting to the gist of gisting. CoRR, 0as/2504.089333 2025.
David Rau, Shuai Wang, Hervé Déjean, and Stéphane Clinchant. Context embeddings for efficient answer generation in RAG. CoRR, abs/2407. abs/2407.09252,, 2024.
Claude E. Shannon. A mathematical theory of communication. Bell Syst. Tech, J., 27(3):379-423, 1948.
Noam Shazeer. Fast transformer decoding: One write-head is all you need. CoRR, ab21191.2211 19121,,,,,, 2019.
Sijun Tan, Xiuyu Li, Shishir G. Patil, Ziyang Wu, Tianjun Zhang, Kurt Keutzer, Joseph Gonzalez, and Raluca A. Popa. Lloco: Learning long contexts offline. In EMNLP, pp. 17605-17621. Association for Computational Linguistics, 2024.
Jiwei Tang, Jin Xu, Tingwei Lu, Zhicheng Zhang, Yiming Zhao, Lin Hai, and Hai-Tao Zheng. Perception compressor: A training-free free prompt compression framework in long context scenarios. In NAACL (Findings), pp. 4093 4108. Association for Computational Linguistics, 2025a.
Jiwei Tang, Zhicheng Zhang, Shunlong Wu, Jingheng Ye, Lichen Bai, Zitai Wang. Tingwei Lu, Jiaqi Chen, Lin Hai, Hai-Tao Zheng, and Hong-Gee Kim. GMSA: enhancing context compression via group merging and layer semantic alignment. CoRR, abs/2505. 12215, 2025b.
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, et al. Kimi k2: Open agentic intelligence, 2025. URL http::s//aav.v..g//////o//////// http://...rr/aaa//5.222.20 20534
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, pp. 5998 6008, 2017.
Chupei Wang and Jiaqiu Vince Sun. Unable to forget: Proactive Interference reveals working mem- ory limits in LLMs beyond context length. In /CMI. 2025 Workshop on Long- Context Foundation Models, 2025.
net forum? ?id YUHksmL8aw
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. In ICLR. OpenReview.net, 2024.
Fangyuan Xu, Weijia Shi, and Eunsol Choi. RECOMP: improving augmented Ims with context compression and selective augmentation. In ICLR. OpenReview.net, 2024.
13

---

Published as a conference paper at ICLR 2026
An Yang, Baosong Yang. Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, et al. Qwen2 techni- cal report. CoRR, abs/2407.10611 2024.
Diji Yang, Linda Zeng, Jinmeng Rao, and Yi Zhang. Knowing you don't know: Learning when continue search in multi-round RAG through self-practicing. In SIGIR, pp. 1305-1315. ACM, 2025.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In EMNLP, pp. 2369-2380. Association for Computational Linguistics, 2018.
Xubing Ye, Yukang Gan, Xiaoke Huang, Yixiao Ge, and Yansong Tang. Voco-Ilama: Towards Computer Vision
vision compression with large language models. In CVPR, pp. Foundation IEEE, 2025.
Chanwoong Yoon. Taewhoo Lee, Hyeon Hwang, Minbyul Jeong, and Jaewoo Kang. Compact: Compressing retrieved documents actively for question answering. In EMNLP, pp. 11424-21439 Association for Computational Linguistics, 2024.
Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou. Long context compression with activation beacon. In ICLR. OpenReview.net, 2025.
Runsong Zhao, Xin Liu, Xinyu Liu, Pengcheng Huang, Chunyang Xiao, Tong Xiao, and Jingho Zhu. Position ids matter: An enhanced position layout for efficient context compression in large language models, 2025a. UBLLttps://arx...or///////
14364
Yiming Zhao, Jiwei Tang, Shimin Di, Libin Zheng, Jianxing Yu, and Jian Yin. Cos: Towards optimal event scheduling via chain-of-schedulingg CoRR, abs/2511. 1. 122111,,,,,, 2025b.
Yunlong Zhao, Haoran Wu, and Bo Xu. Leveraging attention to effectively compress prompts for long-context llms. In AAAL, pp. 66048-26056. AAAI Press, 2025c.
14

---

Published as a conference paper at ICLR 2026
A THEORETICAL ANALYSIS: MIG vs. PURE RELEVANCE
A.1 PRELIMINARIES
Let = (f1,...,, Fn) C Rd be a set of token embeddings, assumed be zero-mean and unit-norm. Let E Rd be query vector, also zero-mean and unit-norm, representngg the target information or label. We use the cosine similarity to measure the relevance between embeddings. Specifically, for any two vectors u, Rd, cos(u, v) u
We define the "relevance" of a token 2; to the query q as:
2
R(i) = cos(s:;, 2) = | Illlll
(10)
Since 24 and g unit-norm, R(i) z. This term captures the linear correlation between the token embedding and the query direction.
We also define the "redundancy" of token 24 with respect set of already selected tokens $ as:
Redundancy((, max cos( 4.))
max ayES
(11)
zyES
> |
Again, assuming unit norm, (edundancy(t.S)) the most similar token already in the maxx, ES x. Tj.- This measures how similar is
A.2 MARGINAL INFORMATION GAIN (MIG)
Inspired by the max-relevance min-redundancy principle (Peng al. 2005) and practical consid- erations in token compression, we define the Marginal Information Gain (MIG) of a token 2; with respect as:
G(i S) R(i) Redundanccy(,,S)= cos(2;;
max cos(31; ayE8
(12)
When considering single token i; in isolation (i.e., S = 0), the redundancy term conventionally set to 0, simplifying MIG to:
G(i) cos(x;,
(13)
However, the power of MIG comes when selecting tokens sequentially or when comparing them against a context. For selection purposes, we are interested in G(i S U {æ;)), which is G(ilS)) = cos q) maxz5es (0,(
A.3 COMPARISON: PURE RELEVANCE VS. MIG
Let X be the set of i tokens with the highest relevance R(i). Let SRE. be set of tokens selected greedily using only relevance R(i). Let SMIG be set of tokens selected greedily using MIG G(i S).
Lemma A.1 (Information Preserved Pure Relevance MIG). Assume selecting of K tokens.
A strategy based on pure relevance R(i) will select tokens that are highly correlated with the query q.
2. A strategy based MIG G(i S) will select tokens that are highly correlated with but are dissimilar to tokens already selected.
Proof Part
is direct: selecting based on R(i) means prioritizing tokens with the highest
cos(1;, 9).
Part 2. follows from the definition of G(i S). By subtracting max cos(2 n.)), ), the score G(i S) is naturally penalized when D; is highly correlated with existing tokens in S. This encourages the selection of tokens that provide "new" information relative to what has already been gathered, as opposed to reinforcing existing information.
15

---

Published as a conference paper at ICLR 2026
Theorem (Superiority of MIG under Redundancy), Let f(S) = g():) be the mutual informa- tion (Dave 2006) between the selected set of tokens S and the target label y. Under the assumption that the token embeddings and the query are sampled from distribution where high relevance often co-occurs with high redundancy among top-relevant tokens (a common scenario in natural language processing), and if [(S) approximately submodular (Bilmes) 2022), then a greedy selection strat- egy (Nemhauser et al. 1978) using MIG (SMIc) is expected to yield a higher mutual information with the target y compared to a greedy strategy using only relevance (SREL).
Proof 2. Let's consider the scenario where there is significant redundancy among the most relevant tokens. Suppose we select K tokens.
Pure Relevance Strategy (SREL): This strategy would greedily select tokens based on R(i)
cos( If several tokens Tegr Ii, are all highly relevant to (i.e., R(im) is large for m = ,.,...,P). but also highly correlated with each other (i.e., cos cos( the pure relevance strategy might select many of these redundant tokens.
large for m # 1),
Under the approximation that mutual information is proportional to the square of the correlation coefficient, T( y) Xx R(1)². When multiple tokens are highly correlated with each other, the addi- tional information they contribute about y diminishes. If x; and x are selected and cos(1;; F3) = T, the information from 2j that is "new" with respect to I; is reduced. In simplified Gaussian set- ting. this redundancy can lead to an information overlap approximately proportional to 7² Cover & Thomas (006). The pure relevance strategy does not explicitly account for this overlap, poten- tially leading to "loss" in effective information captured compared to a strategy that penalizes redundancy.
MIG Strategy (SMIc): This strategy selects tokens based on G(i 1 S) cos(ss; 9)
maxa,es cos(x,, xz3). When considering a token x, that is highly relevant to q (large cos cos( 4)), but also highly redundant with an already selected token zj (large cos(ai, 2z) for some Ij E S), its MIG score G(i () will be significantly reduced. This actively discourages the selection of such redundant tokens.
Therefore, when redundancy is present among top-relevant tokens, the MIG strategy is more likely to select set of tokens that are both relevant to the query and collectively provide diverse, low redundant information. This leads to a better approximation of the true mutual information I(S: y) If f(5) I(S: y) exhibits submodularity (a common assumption for information-thee theereeii objec- tives in feature selection), then a greedy approach based on marginal gain (MIG) is guaranteed to provide a (1 - 1/e) approximation to the optimal set. The MIG's explicit penalization of redun- dancy directly addresses the information loss incurred by redundant features, thus yielding higher expected mutual information.
Conclusion: MIG provides a more robust criterion for token selection and compression compared to merely considering relevance. By explicitly penalizing redundancy with already selected tokens, MIG aims to capture a more diverse set of informative features, leading to higher preservation of mutual information with the target query, especially in scenarios characterized by high token redundancy.
B EXPERIMENTAL EVIDENCE: MIG VS. PURE RELEVANCE
To directly validate that Marginal Information Gain (MIG) better captures token importance than pure relevance, we conduct a diagnostic study on NaturalQuestions, decoupled from the full COMI pipeline. We treat each context segment's 's representative token and evaluate whether its score (MIG vs. relevance) predicts if the segment contains the ground-truth. We use AUC Score (Hanley & McNeil 1982) as evaluation metric.
As shown in Table 5] MIG achieves higher AUC (0.5809) than relevance (0.5423), demonstrating its superior discriminative capability for identifying answer-critical segments.
Furthermore, when retaining top segments by each metric, MIG yields significantly lower redun- dancy. We quantify redundancy as the average pairwise cosine similarity (excluding self-similarrty) among the K compressed embeddings [e.....,ek]:
16

---

Published as a conference paper at ICLR 2026
if S 1,
K
Redundancy Score
ej
K(K 1)
le
|ey
Jaf
Table shows MIG consistently reduces redundancy across different retention ratios.
Table 5: AUC Score for predicting answer-
Table 6: Redundancy of different metrics at dif- ferent retention ratios.
containing segments on NaturalQuestions.
Metric
AUC Score
Retention Ratio
Metrie
Relevance
0.5423
25%
50%
MIG
0.5809
Relevance
0.6839
0.6664
MIG
0.5673
0.5956
C COMPREHENSIVE COMPARISON UNDER Low COMPRESSION RATES
To comprehensively study the performance of different baselines under low compression rates, we evaluate COMI against additional baselines (including SnapKV, LongLLMLingua, and LLMLingua-2) on NaturalQuestions and 2WikiMQA using Qwen2-7B-Instiee B-Itttuuctt the backbone. As shown in Table 7 COMI consistently achieves the highest Exact Match (EM) scores across all compression rates (2x to 32x), demonstrating robust COMI.
Table 7: EM scores under compression rates (2x-32x) on NaturalQuestions and 2WikiMQA.
Compression Rate
Method
2x
4x
8
16x
32x
NaturalQuestions
SnapKV
6.01
5.99
6.25
6.06
5.76
LongLLMLingua
19.81
22.49
19.44
15.07
13.37
LLMLingua-2-BB
15.52
14.39
12.77
10.36
9.60
Activation Beacon
51.83
48.47
41.43
11.68
7.95
COMI
70.58
69.38
64.22
57.48
47.53
2WikiMQA
SnapKV
8.63
8.78
8.29
8.09
1.53
LMLinnu
29.27
25.61
22.42
21.80
21.19
L.LMLingaa-2-arr
27.72
20.87
18.20
18.26
18.81
Activation Beacon
45.75
45.29
45.12
36.59
39.00
COMI
61.80
60.73
57.71
52.85
47.56
D CASE STUDY OF COARSE-GRAINED GROUP REALLOCATION
COMI dynamically reallocates compression budgets based on inter-group MIG. For example, for a context of 256 tokens (A truncated sample from 2WikiMQA) with target 32x compression constraint, the initial group size is 32. MIG-guided reallocation adjusts group sizes to prioritize informative regions. As illustrated in Table Segment (containing key information relevant to query) receives lower compression rate (final size 18 initial size 32), while less informative segments expand. This adaptive behavior ensures information dense regions are preserved with higher fidelity.
17

---

Published as a conference paper at ICLR 2026
Table 8: Example of MIG-guided group reallocation (context length 256, target 32x compression).
Segment Index
3
4
6
7
8
MIG
0.2227
0.0078
-0.1719
-0.4546
-0.5583
-0.3682
-0.3203
-0.2832
Initial size
32
32
32
32
32
32
32
32
Reallo. size
18
22
26
35
39
32
31
30
E SCALABILITY TO ULTRA-LONG CONTEXTS
To evaluate COMI'S ' scalability beyond the 32K context length used in main experiments, we train and test Narrative with contexts up to 64K tokens using Qwen3-4B-Instriue As shown in Table 9 COMI maintains strong performance under extreme lengths: at 16x compres- sion, achieves 22.79, more than double the original prompt (10.69). This demonstrates COMI's scalability to ultra-long input scenarios.
the backbone.
F DATASETS
NaturalQuestions. NaturalQuestions (Liu et al. 2024a) is large-scale dataset designed to evaluate open-domain question answering systems. It is based on real-world Google search queries and uses Wikipedia articles as its knowledge source. Unlike many other NQ are
Table 9: Fl scores on NarrativeQA with 64K max length using Qwen3-4B- Instruct as backbone.
datasets, both the questions and answers derived from authentic user behavior rather than being manually authored. Each question is paired with com- plete answer, which can be either a short text span from a Wikipedia page (the "short answer") or a longer text passage (the "long answer"), enabling the simultaneous assessment of a model's precise extraction ability and its comprehension of long documents. The specific version utilize contains 20 documents in total, with only one being the ground-truth document and the others serving as distractors.
Compression Rate
Method
16x
32x
Original Prompt
10.69
10.69
COMI
22.79
20.80
HotpotQA. HotpotQA (Yang al. 2018 is dataset for multi-hop question answering. Its unique characteristic is that each question necessitates finding and synthesizing information from multiple Wikipedia articles. The questions often involve several entities and facts, requiring models to per- form cross document reasoning and linking to construct a coherent and complete answer, rather than simply extracting single piece of information from one document.
2WikiMQA. 2WikiMQA (Ho et al. 2020) is another dataset specifically engineered for complex, multi-hop question answering. Similar to HotpotQA, it requires models to reason and integrate information from multiple Wikipedia documents. However, its questions typically involve more complex logic and inference chains, challenging models to not only identify relevant facts but also understand their relationships (e.g., such comparing, contrasting, or inferring causality) to generate an accurate response.
MultiNews. MultiNews Fabbri al.) 2019) dataset for multi-document summarization. contains large collection of news clusters, with each cluster composed of multiple articles reporting on the same event. The objective to teach models how to extract key information from these disparate reports and synthesize it into a concise, coherent, and low redundant summary. This task requires models to identify redundant information, integrate knowledge from multiple sources, and generate fluent text.
NarrativeQA. NarrativeQA (Kociský et al. 2018) a dataset designed to evaluate machine com- prehension and summarization capabilities. It includes full-length novels and movie scripts from Project Gutenberg, paired with queries often require deep understanding of plot development, character relationships, and event
suman-generater questions and answers. The
18

---

Published as a conference paper at ICLR 2026
sequences, compelling models to perform sophisticated inference over long-form narratives to pro- duce accurate responses.
G LANGUAGE MODEL USAGE STATEMENT
During the preparation of this manuscript, we utilize a large language model as a writing assistant. Its primary role is to refine and polish our paper, including the descriptions of our methodology and the presentation of mathematical derivations. This done to improve the overall clarity, precision, and readability of the paper. All core ideas, experimental designs, and results are original work of the authors.
19


Reviews:
[{"review_id": "4nJb0gcz2v", "reviewer": "Reviewer_Locn", "summary": "This paper addresses the issues of slow inference, high computational cost, and information redundancy in large models under long-context scenarios by proposing COMI: a coarse-to-fine compression framework based on Marginal Information Gain (MIG). Building upon GMSA, the method introduces two key improvements:\n\n1.Adaptive compression rate allocation for different segments using MIG as a metric.\n\n2.Weighted token merge within groups based on MIG, suppressing redundancy while preserving relevance.Experiments show that COMI significantly outperforms existing methods even at a 32× compression rate.", "questions": "1.The article compares methods under the same compression rate, but Section 5.4 shows that COMI's FLOPs at 32× compression are higher than those of GMSA. Have the authors attempted to compare the accuracy of different methods under the same FLOPs?\n\n2.In the experimental Section 5, Figure 4 only shows the performance of the Activation Beacon method and COMI at 2×, 4×, and 8× compression rates. How do other methods perform under these conditions, and does COMI still achieve the best performance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.Addresses redundancy among relevant tokens by incorporating a penalty score for related but redundant tokens, effectively reducing token redundancy.\n\n2.Adaptive compression rate allocation across groups avoids applying the same pruning rate to both high-relevance and low-relevance groups, making the method more reasonable.\n\n3.Weighted token merging within groups based on MIG scores reduces redundancy while ensuring relevance and preserving semantic diversity.\n\n4.Experiments demonstrate the effectiveness of COMI across different models and task types.", "weaknesses": "1.Insufficient clarity in figures: The layout of II/I/III in Figure 3 may cause reading difficulties. It is recommended to improve or enhance the labeling.\n\n2.Insufficient comparative experiments under low compression rates: Comparative experiments under low compression rates are only conducted with the Activation Beacon method. It is recommended to include more methods in the tests.\n\n3.Lack of controlled experiments under the same FLOPs: Although the authors compare accuracy under the same compression rates, they do not compare the accuracy of different methods under the same FLOPs. Table 5.4 shows that COMI's FLOPs at 32×compression are higher than those of GMSA. It is recommended to supplement performance comparisons under the same FLOPs to more fairly reflect the efficiency-effectiveness trade-off.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YmovsJIFLj", "reviewer": "Reviewer_fQJU", "summary": "This paper introduces COMI, which applies Marginal Information Gain (MIG) to compress long contexts. It first reallocates compression budgets across segments via inter-group MIG, then merges tokens within each segment using intra-group MIG weights. Experiments are conducted to demonstrate the effectiveness of this method.", "questions": "See the weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The methods part of the paper is written clearly.\n2. The structure of the paper is well organized.", "weaknesses": "1. The proposed method seems to work only for the encoder-decoder architecture, but this is not the mainstream architecture nowadays. There is no motivation discussion to clarify why the authors would like to design a compression method that does not work for the most popular decoder-only structure.\n2. There is no analysis of the compression rate pattern. The compression rate is determined dynamically via MIG; however, the paper does not provide any analysis to illustrate the pattern of the compression rate.\n3. There is no comparison of the compression rate between the proposed method and other methods. I think this is important to eliminate the concern that performance improvement may come from the relatively low compression rate compared to other methods.\n4. The efficiency analysis needs more details. The proposed method separates the compression and generation stages, but there is no information on which stages are used in Table 3.\n5. There is no limitation part to discuss the potential improvements of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SdHvht0IPE", "reviewer": "Reviewer_Nnax", "summary": "This paper introduces COMI, a context compression framework that tackles inefficiency and redundancy in long-context LLMs. Its core contribution is the Marginal Information Gain (MIG) metric, which guides compression by balancing query relevance against information redundancy.\n\nThe MIG metric prioritizes information that is both salient (relevant) and unique (non-redundant).\n\nCOMI operates in a coarse-to-fine process:\n\nCoarse-Grained Group Reallocation: Dynamically allocates the compression budget across large semantic groups based on their inter-group MIG.\n\nFine-Grained Token Merging: Uses intra-group token-level MIG as SoftMax weights to merge tokens, preserving key information within each group.\n\nKey Results:\nCOMI was evaluated on QA and summarization tasks (e.g., NaturalQuestions, MultiNews) at high compression rates (up to 32x). Using a Qwen2-7B backbone at 32x compression, it achieved an improvement of approximately 25 points in Exact Match (EM) on NaturalQuestions, significantly outperforming baselines. Ablation studies also confirmed the effectiveness of both stages and the redundancy penalty.", "questions": "Details of Figure 1: Could you provide more detail of the Figure 1? Also, I think It would be better to measure the redundancy of compression within the existing compression methods rather than LLMs attention, which can more clearly illuminate the limitations of current methods.\n\nImpact of segment size: What is the size (e.g., number of tokens) of the \"equal-length\" segments used for coarse-grained reallocation? How was this size chosen, and how sensitive is the model's performance to this value?\n\nTraining Fairness: Were all baseline methods compared under identical training data and compute budgets to ensure a fair comparison?\n\nCompression Rate Flexibility: The method appears to be trained for and evaluated at fixed compression rates (e.g., 16x, 32x). Is it possible for COMI to dynamically determine an optimal compression rate based on the query or content complexity, rather than relying on a fixed target?\n\nQualitative Analysis and Interpretability: Can qualitative examples or case studies (including failure cases) be provided? It might be insightful to approximate the results of compression into interpretable forms (e.g. natural language) and compare with other baselines.\nThis would help demonstrate how COMI retains important information and removes redundancy in practice.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Clear Motivation: The paper's motivation is clear. It identifies that LLMs fail to address the high semantic redundancy found among query-relevant tokens, leading to suboptimal attention allocation. The pilot experiment provides empirical evidence for this claim. The redundancy and semantic overlapping in the compression units have not been covered mainly in this domain. \n\nStrong Empirical Results: The method demonstrates significant performance improvements over existing baselines, especially at high compression rates.\n\nEfficiency: The paper reports reasonable latency, suggesting the method is practical and achieves its strong performance.", "weaknesses": "### Applicability to Stronger Long-Context Models: \n\nWhile COMI effectively boosts the base models' ability to digest context with reasonable latency, it notably outperforms even the original prompt baseline. This suggests that the chosen base models (LLaMA-2-7B-chat, Qwen2-7B-instruct) have weak innate long-context understanding (also widely known as the \"lost-in-the-middle\" problem).\nTherefore, I want to see whether COMI can still provides benefits when applied to current models (e.g. Qwen3-4B-Instruct, ...) that are already equipped with long-context capabilities. For such models, simply using the full, uncompressed prompt might be a more direct and effective solution for these QA tasks. While this might be a bit rigorous standard for a compression paper, demonstrating gains on these current models would be necessary to significantly support and expand the paper's claims.\n\n### Performance at Scale (Context Length): \n\nIt seems that the experiments are capped at no longer than 32k context. This is relatively limited given that modern models are pushing to 1M+ tokens. How does the method scale to contexts significantly longer than 32k? A stress test on much larger inputs is needed to verify its scalability.\n\n### Risk of Single-Token Representation for Budgeting\n\nThe coarse-grained reallocation stage determines a segment's entire compression budget based on the MIG score of a single representative token (the one with max query relevance). I am concerned about the validity of this proxy. If other, non-representative tokens within that same segment also contain information crucial to the segment's overall value (e.g., less relevant but also far less redundant info), their contribution is ignored during this critical budgeting step. This could lead to an inaccurate estimation of the segment's true aggregate MIG (relevance and redundancy) and result in a suboptimal budget allocation. I wonder if this simplification is problematic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g5NXF8Jo3h", "reviewer": "Reviewer_RNvR", "summary": "The paper addresses the computational inefficiency and semantic redundancy of long-context LLMs, arguing that existing methods overlook redundancy among retained tokens. The proposed COMI framework introduces Marginal Information Gain (MIG)—a metric that balances query relevance with low redundancy—to guide its two-stage compression process. This process includes Coarse-Grained Group Reallocation for adaptive budget distribution across context segments and Fine-Grained Token Merging for a weighted combination of unique tokens within those segments.", "questions": "1. Validation of the Novel Metric (MIG)\n>The effectiveness of the core technical contribution, the Marginal Information Gain (MIG) metric, is currently inferred from final performance scores. We consider the direct empirical validation of this novel metric to be paramount for the rigor of the submission.\n>Could the authors provide a dedicated diagnostic analysis to verify that the computed MIG level correlates better with a token's true importance (e.g., its presence in the ground-truth answer) than a pure relevance score does?\n\n2. Missing Diagnostic Evidence for Core Claims\n>The paper claims success in retaining query-relevant information and removing semantic redundancy.\n>To support these crucial claims, could the authors provide additional quantitative analyses demonstrating the effectiveness of the proposed methodology in both areas? This evidence should move beyond EM/F1 scores to examine the characteristics of the compressed context itself.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The main idea of establishing a compression criterion based on Marginal Information Gain (MIG), which explicitly rewards relevance and penalizes redundancy, is a valuable contribution to the field. This addresses a key weakness of previous work that depends only on relevance.\n\nThe paper validates COMI across a robust set of benchmarks spanning different task types, including single-hop QA, multi-hop QA, QA on long narratives, and summarization, using two modern LLM backbones (LLaMA-2-7B and Qwen2-7B), demonstrating its generalizability and robustness.", "weaknesses": "**1) Lack of Direct MIG Metric Validation**\n>The paper's entire foundation rests on the Marginal Information Gain (MIG) metric, which influences two key, complex decisions: group reallocation and token weighting. The authors rely on circular reasoning, assuming that the improvement in end-to-end performance (EM/F1 scores) implicitly proves that MIG is effective.\n>However, they only provide theoretical justification (Appendix A) and indirect, final results. They neglect to conduct an essential diagnostic experiment to verify that the MIG score actually indicates higher informative value and lower redundancy in a measurable, ground-truth context. Specifically, they do not demonstrate whether a token's MIG level correlates better with its true importance (such as being part of the ground-truth answer) than a simple relevance score.\n>To improve this, the authors need to perform an independent, diagnostic validation of MIG prior to embedding it into the complex COMI framework. This should include a statistical comparison (like AUC) showing MIG's ability to predict critical tokens better than mere relevance.\n\n**2) Missing Diagnostic Evidence for Core Claims (Retention and Elimination)**\n>The paper's two main objectives are to successfully retain query-relevant information and remove semantic redundancy. However, the current evaluation is inadequate because it only reports final LLM outputs (EM, F1), which serve as indirect indicators of compression efficiency.\n\n> **Evidence for Retention**: The paper never reports on the efficiency of its selection process. It is unknown what percentage of the actual critical tokens (those required for the ground-truth answer) are physically retained in the compressed context compared to baseline methods. The high final EM score is thus only an indirect indicator, not a definitive measure of success in retention.\n\n> **Evidence for Redundancy Elimination**: The most novel part of the MIG metric is the redundancy penalty, yet the authors provide no internal analysis to demonstrate that the final compressed context is actually less redundant or more diverse than contexts generated by relevance-only methods. Without a quantitative measure of diversity within the compressed context itself, the claim that MIG effectively \"eliminates semantic redundancy\" remains unsubstantiated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)